Did Dr. Hannibal Lecter like Clarice or was he attracted to her?

Yes, Hannibal Lecter sees a lot of things in Clarice:

1. Someone who is smart, but not rude. (Rudeness is Hannibal's biggest pet peeve).
2. Someone who has suffered in a way that he has suffered (childhood trauma, losing family),
3. And eventually, someone who may be able to house his dead little sister's conscience.
4. And someone to love, who can also heal his childhood trauma.

In Silence of the Lambs (book and film), it isn't perfectly clear what it is exactly that draws Hannibal to Clarice and even if he is particularly drawn to her or just screwing with her head. But the religious depictions and time dedicated to her become much clearer in the next two novels Hannibal (which takes place seven years after SOTL) and Hannibal Rising (a prequel origin and revenge story set during Lecter's childhood and late teenage years).

The Books (original source material):

In these novels, it becomes clearer that Lecter and Clarice both suffered some traumatic childhood experiences that involved losing someone they loved. For Clarice, it's her father. For Hannibal, it's his little sister.

The Hannibal novel, however, has a lot going on that isn't really explained. Hannibal somehow comes to believe in necromancy; that Clarice is a perfect vessel for his little sister's (Mischa's) consciousness.

And so I came to believe, " Dr. Lecter was saying, "that there had to be a place in the world for Mischa, a prime place vacated for her, and I came to think, Clarice, that the best place in the world was yours." -Hannibal pg 535

Lecter goes through great pains to try and psychologically drive Clarice to bend her to his will, but ultimately comes to realize he has no control over what he created and Clarice declares that the best space for Mischa would be his own.

"It occurred to Dr. Lecter in that moment that with all of his knowledge and intrusion, he could never entirely predict her, or own her at all. He could feed the caterpillar, he could whisper through the chrysalis; what hatched out followed its own nature and was beyond him." - Hannibal (about Clarice) Pg523

Normally, one might assume that Hannibal wouldn't accept this, but oddly Hannibal does (because he's in love), and the two go off to Buenos Aires together. From Clarice's perspective, despite the manipulation, there may be subtext from Thomas Harris about Clarice transforming into a woman, as there are some erotic lines about nurturing Lecter from her breasts.

In the film version, Lecter also puts Clarice in a rather expensive designer dress, as he helps Clarice with her misogynistic nemesis at the FBI, Paul Krendler. These details are important, as Clarice is thought of as homely and without taste (and taste is something Hannibal cares about greatly), which could play back to how Lecter remembers his toddler sister, being sort of imperfect and messy.
At any rate, the final lines of the Hannibal novel are very suggestive that Clarice is where she wants to be and Lecter is now beginning to heal from his past, because of this romantic relationship...

"Occasionally, on purpose, Dr. Lecter drops a teacup to shatter on the floor. He is satisfied when it does not gather itself together. For many months now, he has not seen Mischa in his dreams.

Someday perhaps the cup will come together. Or somewhere Starling may hear a crossbow string and come to some unwilled awakening, if she indeed even sleeps. We'll withdraw now, while they're dancing on the terrace--..." -Hannibal pg 544

Hannibal Film Adaptation/Novel Comparison:

However, the film version of Hannibal takes things in a slightly different direction, as all knowledge of Mischa is omitted from the film and Clarice Starling is not psychologically manipulated into the person she becomes at the end of the novel. In fact the film's ending is dramtically different for this reason.

But as Directer Ridley Scott has stated, the film (like novel) features themes of romance, punishment, and retribution and where a conversation Lecter as with Allegra, is a reference to how Hannibal feels about Clarice Starling

Scott openly admits to a "romantic thematic" running through the film. He told CNN that: "Hannibal was quite a different target, essentially a study between two individuals. Funny enough, it's rather romantic and also quite humorous, but also there's some quite bad behaviour as well." During the opera scene in Florence, Lecter attends an operatic adaptation of one of Dante's sonnets, and meets with Detective Pazzi and his wife, Allegra. She asks Lecter, "Do you believe a man could become so obsessed by a woman after a single encounter?" Lecter replies: "Yes, I believe he could ... but would she see through the bars of his plight and ache for him?" This scene, in the film, is one which Scott claims most people "missed" the meaning of. It was in reference to Starling—to their encounter in The Silence of the Lambs.[19] The New York Times, in its review of the film, said Hannibal, "toys" with the idea of "love that dare not speak its name"

In addition both versions feature a scene of Mason Verger setting a trap for Hannibal by capturing Clarice, as Verger figures out from listening to Hannibal Lecter's recorded tapes from his time in the insititution, that he does seem to care greatly for Starling. So he has Starling kidnapped to use as bate! (See Picture Above)

But even outside of what both the film and book kept the same, the film still retains a great deal of subtext back to the novel, including that in the film adaptation's final acts, Hannibal still demonstrates his love for Clarice. (And maybe even more so than the book's)

"Tell me Clarice, would you ever say me, Stop. If you love me, you'd stop."

Hannibal leans into kiss Clarice, but she handcuffs him, as the FBI are on there way to their location. Hannibal finds a butchers knife and tells Clarice, "This is REALLY going to hurt." Hannibal cuts off his own hand and escapes!

As for Clarice's potential affection for Hannibal Lecter, the film remains somewhat ambiguous about it. She never does anything blatant to show she cares, and she even denies having feelings (although you can read that in more than one way), but generally she keeps to a very moral center of what she believes her duty is, which goes back to novels' passages about her father, a police officer that was killed on duty.

The final scene of the film also shows Hannibal feeding some leftover remnant's of Paul Krendler's brains to young boy. As disturbing as the scene is, it calls back Hannibal's feelings towards the innocent and his own childhood.