A common misconception about collecting graveyard dirty is that it symbolizes the occupation or skill of the person you collect dirt from, but this isn't the case in traditional rootwork.
In actuality you are hiring a spirit to do a job for you. But what if you collect dirt for instance from a bank for money problems. Are you hiring the of that place to do a working for you? Are all places you collect things from a form of hiring a spirit?
Hmm... That is indeed a thought to ponder! personally i can see the symbolic connection, being that places carry the energy of their function. So a place like a bank, which is filled every day with people working money, thinking about money, making and spending money, that the area could pick up that energy that could then be connected to in such a way.
In relation to Voodoo though, I am not sure it if would apply in the same way. From what I am aware, Vodou/voodoo is focused on connecting to and employing the help of ancestors and other living spirits. Physical people or animals that have crossed over to the spirit side are typically what the practitioner is making use of. So while the dirt from a bank might connect you to the energies related to money and money-making, it might not connect you to a spirit per-se. Just the energy of the building itself. To connect with a -spirit- related to money, by Vodou you would likely be best off with the dirt of a banker's or investor's grave, as that would connect you with a living spirit instead of a field of energy.