I never thought I would think so hard about brick dust.

But the discussion is interesting, because it's about the properties of things, and the question of whether the properties are physical, or metaphoric, or both. Which isn't that easy a question for some of us, and not easy with this particular type of magic. I'm not even sure it's because we were raised on horror movies so much as that a lot of us were raised away from nature. So the nature of natural things is mysterious (metaphoric), and these days when parents don't let their kids play outside in the mud anymore, imagined to be dangerous. But given that such innocuous things as bricks also have magical uses by (or against) dangerous people, things, even thoughts, the questions make sense.
But also, there might be conflicting folklore at work here, not just contemporary media-infused paranoia.
Miss Cat, I used to live on the west coast, and I've been through northern California lots of times. And you're right, it would be real work finding old bricks from mental hospitals. I don't even know where you'd go, and it would all be hilariously pointless. But the northeastern US is lousy with them. I live in the sticks too, and there's a gorgeous spooky old gigantic sanatorium about 10 miles from here just crumbling away, and another state hospital an hour from here, with similar architecture to the infamous Danvers State Hospital near Salem MA. Danvers was Lovecraft's model for Arkham Asylum. Such places, built along the Kirkbride Plan, were the latest in 19th century mental health facilities. And people suffered in them, and they are freaking haunted --at the level of the architecture itself. I wouldn't use bricks from them either. The American northeast is a haunted, decaying, 19th century physical landscape in a way that the West Coast isn't because it's too new, and these buildings -- and even the pieces of them, are miserable and lively in a way that they may not be in other places.
So, people learning southern Hoodoo, which comes from a very different region, may well come to it with the place/idea of the northeast, which is more familiar to them, and just as real in its own way.
Raising overthinking it to epic levels? Moi?