Did Jabba the Hutt force himself on the captured Leia?

The implication is that they didn't, at least according to the film's junior novelisation.

We see Leia and Jabba in his apartments. He licks her, then sends for some more appropriate (and revealing/humiliating) clothing for her to wear at his next party.

“I’m sure,” Jabba slobbered. His tongue lolled out of his wide mouth and brushed against the side of her face and clothes.
“Ugh!” Leia said, recoiling.
C-3PO trembled and looked away. “Ohhh, I can’t bear to watch,” he cried. Jabba thought Leia had a pleasant flavor, but he didn’t care much for her drab, dusty clothes. He ordered Bib Fortuna to fetch some garments that were more to his taste….

He then has his party and, tired from his exertions decides to have a nice snooze. We get a third-person POV of her thoughts. She's certainly not happy but there's no indication that she was sexually assaulted.

Leia, eyes closed, lay slumped beside Jabba’s slumbering form. She had replaced Oola as Jabba’s slave dancer, and was worried about the acrobatic dances she might have to perform. She wore a collar around her neck that was secured to a long chain. She also wore a skimpy costume that left very little of her unexposed to view. Salacious Crumb — who remained awake — leaned over from his resting spot within the curve of Jabba’s tail and peeked at Leia’s bare stomach.

Although it's theoretically possible that a new source of information could 'de-canonise' these events, given the recent furore over the decision to retire the "Slave Leia" outfit from future merchandising, the chances of Disney contradicting this and confirming that she was raped by a giant space-slug are basically zero.

Sexual assault is an act in which a person intentionally sexually touches another person without that person's consent, or coerces or physically forces a person to engage in a sexual act against their will.

Jaba forcing Leia to wear revealing clothing, restraining her, and then making advances such as trying to lick her body, were unwanted sexual advances.

So most definitely Jabba forced himself on Leia, against her will, and without her consent.