@ flarepaw could of been a spirit, or your mind playing tricks, if you were alone or scared for instance.
@ magiclover66, is it throughout the day, or when you're in certain places? throughout the day all over town, it could be a spirit. if it's in one place, it still could be a spirit, but like WhiteRav3n said, check for any drafts. the house could be drafty, there could be an air duct, or perhaps an open window. try speaking with it, but be sure you rule out all the logical ends like the wind.