Is Fantastic Beasts and Where to Find Them understandable without watching any other Harry Potter film?

There is no need to watch any Harry Potter movie to understand Fantastic Beasts.

Here is why:

Fantastic Beasts And Where To Find Them is not a prequel to all eight Harry Potter movies. The Harry Potter franchise is set in the '90s while Fantastic Beasts is set in 1920's.

No one from Harry Potter exists in this era, except Albus Dumbledore. And there are some references about magical creatures that can be found in Harry Potter books/movies but you don't have to worry about it because you won't miss anything here nor you will feel lost during the movie. But you will be entering a new established universe.

A Bustle article goes into greater detail as to why it's fine for newbies.

I think you can watch this as a stand alone movie. Apart from a few references to the Harry Potter world in the movie (name drops of Albus, Hogwarts - total of around 10 small references through out the entire movie) etc., rest of it is completely new. Its set in the 1920's era. Before Harry Potter started.

So no, you don't have to watch the Harry Potter movies to understand this one. It's a new experience even for HP fans (you would know the beasts they show in the movie if you read Fantastic Beasts library book before)

Spoilers below as requested.

Hogwarts keeps coming up - also a my-school-is-obviously-the-greater chat comes up with the main protagonists --- Gellert Grindelwald comes up. You can even see the back of his head right in the start --- Dumbledore's name comes up. One character asks Newt why Dumbledore favors Newt.--- Nifflers, Bowtruckles, Occamys, Diricrawls, Demiguise etc., beasts from 'fantastic beasts and where to find them' book from Hogwarts Library appear in the movie --- Seeker, Chaser reference comes up..

and few more references which I can't exactly recollect.

Yes, it is understandable without having watched the other HP films, but you might want to pre-explain certain aspects of JKR's wizarding world - about the separation of the magical & non-magical worlds, how the magical world has it's own government (specifically the Ministry of Magic), Aurors, Muggles, magical schools, the wizards' newspapers with their moving images and the dark/light wizards - you could use a Star Wars analogy here :-).

Mostly

The movie does not explain the Deathly Hollows symbol, or that Grindelwald used it as his own symbol. So anyone not familiar with the storyline from the HP books/movies will completely miss the significance. Other than the opening credits and one brief mention, Grindelwald doesn't come much. So Graves being Grindelwald is a bit out of left field.

Other than that, everything is pretty well contained and explained. The rest of the HP universe references do not hinder the plot.

Yes, I'd say so. I've watched the movie. The difference is Harry Potter is in Britain, and Fantastic Beasts and Where To Find Them is in America. Besides the no-maj and muggle thing, I guess you're OK. Also, no-maj and muggle mean no magic.