Critical thinking and educated judgement are the keys to this conundrum. I would love to say it is 'common sense' but that is a phrase I have never really liked. After all it is only 'common' to those who already know it.
For the education part; Magic is a function of will, visualization, and symbolism. It works with the principles observable in nature, not against them.
The ideas of magic within entertainment- be it TV, movies, board games, books, video games, or table-top rpg's, all rely on the idea of it being a tool or story trope that allows someone to bypass the basic rules of the world. They define magic as something that let's you do something you are not supposed to be able to do. That's what makes it magical/fantastic. It is great for story telling, creates fuel for a variety of forms of drama, and it is, of course, entertaining. But that's because it is fantasy. Basic ideas brought to their extreme of scope to inspire a sense of stakes, power, and escape. I look at this stuff like candy. Fun, but of no substance.
One thing to bear in mind, is that in history the shaman, cunning woman, medicine man, priest/ess, druid, etc all had a role within their community that was centered on helping it survive. It was a position considered to be of education (for the time), and much of their 'magic' was understanding. Knowing the seasons, when to plant or harvest, reading the weather to know when storms were coming. Burial rights for the dead. Being a mid-wife. Medicine and herbcraft for treating the sick or injured. The next function was to be the story teller and keeper of the history of the community. To tell the stories of the past, and the stories and lore of the gods, as a way of passing on the moral ideals and social expectations of the people.
And finally, the last role played is to be intermediary between the physical world, and the spiritual world/the gods. They would lead prayer and sacrifice, interpret omens,and try to give appeasement and/or gain favor from the gods for good weather, rich harvest, preventing of disease, and victory in battles.
The ideas of magical practice have evolved over time somewhat, and some of the roles of the practitioner have changed. A couple have almost disappeared entirely. But understanding the roots of a practice helps one to gain a sense of perspective and knowledge that will lead you to that elusive common-sense judgement.
Here are a few principles I have picked up along my way;
Like attracts like. This is the basis of symbolism.
Movement occurs in the direction of least resistance. Basically, physics and application of forces also applies in principle to magic. You can start a landslide with explosives, research, drilling, and planning. Or you can start one by throwing a pebble once the time is right.
All things begin with the Self. Be the change you wish to see.
And similarly to the former, the key to discovery lies not in seeking new landscapes, but in having new eyes.
There are generally two ideas about what is fluffy.
The newer idea is that a "fluffy" practitioner is someone who pursues unrealistic things, into the realm of expecting to make pure fantasy a reality. I mean things like growing wings, becoming a different species or hybrid, levitating, X-ray vision, and I'm sure you get the drife.
The term actually originates from the longer term, "white light fluffy bunny," talking a bit more about people who seemingly deny any negativity even exists, and insist that being positive about everything is the way to be. They'd deny that there's any evil, negativity, and deny those aspects of themselves, even. Like, "Oh, you're not having a bad day! Just remember the white light and all goodness flows through you and suddenly it should get better!" and theybelieve it works that way. In a sense it's also unrealistic fantasy, but it's a different kind.
As others have said, critical thinking is among the skills needed, but really it's just keeping in mind that sometimes you can't change what is, and some things are simply not realistic. Sometimes circumstances are more bearable depending on how you respond to them rather than what they are in the first place, and I think that can apply to a lot of things:
I wish I didn't have to drive to distant places. Sure, but that doesn't mean you can learn to fly.
I dislike this feature of my body. Yes, but that doesn't mean you can grow a set of wings to somehow become angelic or whatever it is you think it will make you.
I don't like my job. But That doesn't mean I can make everyone else around me behave how I want them to.
I'm sorry, I haven't seen the term "fluff" in so long, I actually got a little excited like "oh, wow, am I in high school again? What a time warp!"
Anyway, regarding your question. The above posters make excellent points. Study "the basics" of magick, so you at least understand how it works. Then, when you come across a spell, ask yourself if it seems possible without magick. [You can ask your crush out, therefore a love spell should work. You can't fly, therefore a spell claiming you can probably doesn't work] Magick is an energy found in nature that brings natural change, so things like love, luck, health, wealth, these are all things you can achieve on your own. Magick charges the energy around you or a situation to attract the desired change easier than without a spell. Once you've narrowed down the working categories, it becomes practice and understanding how/why spells work. You might find a love spell, but all the ingredients aren't for love, meaning the spell won't work because of design. You can also rely on older practitioners to guide you and help you with your first couple spells until you become more familiar and can weed out the bad ones.
How to know your spell is fluff? Avoid the fantasy and trick sections of the spell section for now. There are a ton of fluffy spells in those sections. I feel you confuse "fluff" and "flawed" a "fluff" spell is usually "this spell will make you a half werewolf, vampire, bunny cat with rainbow wings" whereas a flawed spell would be "attract your true love while casting during the waning moon." The spell itself would work, provided you change one or two little things in the design [love spells to attract love are typically done in the waxing moon as the energy is flowing towards you, and the waning is when you send things away as the energy flows out] In this instance, again, it's understanding how magick works, and asking practitioners for advice [but I would be careful whose advice you use, as there can often be a dogpile method in Paganism where people yell above each other swearing their way is the right way. Honestly, apart from what magick can/can't do on the physical, everyone else's opinions are simply advice and not facts you must obey. Some paths have specific rules, but if it's not resonating with you, you don't need to follow it]