Why is the movie titled Gothika?

Gothika is just a made-up word that you won't find in the movie. You might find different definitions of this word but none of them are verified. However, it seems this word is derived from the word "Gothica", i.e. related to Gothic styles.

From IMDb with emphasis of mine,

Most likely, gothika is merely an affectatious spelling of gothica and is similar to other "-ca" words such as "galactica" (things pertaining to the galaxy) or "erotica" (things pertaining to erotic media). Thus, gothika can best be described as "things pertaining to Gothic styles," as in architecture (i.e., the Woodward prison resembles a creepy Gothic castle), art (the Anima Sola), the supernatural, the macabre, and preoccupation with death. Even the movie's photography, with its focus on blues and blacks, is suggestive of the dark and gloominess associated with Gothic literature.

From Allocine, a reference about movies in France,

Le titre Gothika renvoie clairement à la tradition de la littérature gothique. Né à la fin du 18e siècle en Angleterre, ce style met en scène des personnages prisonniers de lieux isolés et affrontant des phénomènes macabres et mystérieux, voire surnaturels.

Here is my own translation:

The title Gothika clearly refers to the tradition of Gothic literature. Born at the end of the 18th century in England, this style stages characters imprisoned in isolated places and facing macabre and mysterious phenomena, even supernatural.

This fits with the atmosphere and themes of the movie. I couldn't find if this is Allocine's guess, or if it comes from an interview of the director Mathieu Kassovitz (who happens to be French)