Depending on where you live, what stores are near you, dollar stores are almost always a good place to find candles. For example, they often have four-packs of emergency candles -- plain white pillars, about 3/4 inches wide, about 4 inches tall -- for a dollar. They also sell tall tapers for a dollar per pair, almost always in white, red, and/or ivory. If those are short, they seem to never quite run out of little votives and scented candles. And being so inexpensive, they're often not strongly scented, which leaves them fairly generic smelling.
Otherwise, I'd suggest online. I was just eyeballing a box of 72 Shabbat candles (for traditional Jewish worship) for about $10 on Amazon. That's just under 14 cents per candle! It may be a little out of your budget at the moment, but if you save up a bit (and if you don't have Prime, saving a bit more to order enough, and buying a few other things you want or need, to get free shipping may help) it would keep you in candles for a little while, at least.
If there's an Ikea near you, I would definitely suggest a stroll through their candle section. I have yet to find tea lights for a better price than when I used to live conveniently close to an Ikea. But they also have a lot of other kinds of candles, for not a ton of money.
Otherwise, I've seen scented candles pretty frequently at buyout stores like Ross and Marhsall's, and of course there's always good old Walmart.
I know not everyone is a fan, but ordering then from the Dollar Tree is SUPER cheap. You can even get them in multipacks making them even cheaper. You can either have them delivered to a Dollar tree location for free or have them delivered for $4.95 flat rate, which is not bad at all considering how cheap you are getting your candles.
I also save all of my wax (unless a spell calls for something particular to do with it) and then I melt it all down in a crockpot I use just for this, or if its a small batch I use an old glass candle container placed on a candle warmer and then melt down chunks that way. Then it is just using thick string braid I use for the wick in glass containers that I fill up with the melted wax. While I don't get to choose the color of these, I love using them because I feel like they are filled with the intent and energy I have infused them with over their recycled lifespan. I do a little cleansing and charging chant for good measure, of course, but this is a practice that is close to my heart...and also saves money!