Parasitic oscillations are one of the four typical causes for emissions in EMI/EMC problems. Try to reduce the gain or break the feedback and the problem could be solved at low cost.
For many applications, especially in digital designs, you can see decoupling networks composed of several different (big and small) capacitors in parallel. But, sometimes this technique can be dangerous.
You can minimize your EMI/EMC and SI/PI problems by working as slow as possible. This is very well known advice from many experts, books, and seminars: you can minimize or solve electromagnetic interference (emissions/susceptibility) and signal or power integrity problems working as slow as possible.
When designing an EMI/EMC filter the orientation relative to source and victim is critical for high effectiveness. Can you save components in your filters?
If you are a novice designer of electronic circuits, this is one of the best pieces of advice I can give you from my experience in EMI troubleshooting and training.