Apprentice magician kidnaps people to turn them into human-shaped building blocks [closed]

I have a memory of watching a movie on TV when I was a kid. I wasn't familiar with anime at the time, but looking back I believe it was anime. I would have watched it sometime between 1982 and 1986 on broadcast TV in Cleveland, OH.

The plot focused on a man who was an apprentice magician or sorcerer. His master made him kidnap people, who would then be transformed into roughly human-shaped building blocks that the sorcerer then used to expand his castle. The apprentice knew what he was doing was evil, but went along with it to protect his little sister. He bides his time, covertly working to ultimately defeat his master and save all the people who have been taken and have used as building material.

That's the gist of the movie as I remember it, but I'm aware that memory is imperfect. Hopefully it's enough to be identifiable.