Look for costumes made from 100% natural materials such as cotton or silk. Hemp and burlap are child and Earth-healthy and make great costumes.
Wool is naturally flame retardant (as opposed to many costumes which are sprayed with chemicals to keep them "safe") and is naturally water repellent (which comes in handy if you live in the Northwest!).
Dress infants in themed cotton onesies with matching hats instead of in polyester costume buntings.
If you do buy a costume off the rack or from thrift or consignment stores, be sure to wash well before allowing your child to wear it for the night. A few days in the freezer or exposure to high heat (e.g. in the dryer) will kill lice and bed-bugs.
Costume jewelry and accessories such as pumpkin treat carriers often contain chemicals such as lead (a neurotoxin). If you are unsure as to whether an item is safe, have it checked by an expert (do an Internet search for "lead testing" in your area), use an at-home lead testing kit, check to see if the item is included on this registry of unsafe items, or text the word KIDS and the item name or brand to 30644 to find out whether or not the product is known to have heavy metals. If you are unsure or suspect that the item may be unsafe, find an alternative.
Avoid masks that cover the face and accessories that go in the mouth, such as fake teeth. They may contain heavy metals and chemicals (called phthalates) that make the plastic pliable.