Do I just swipe banners? Swipe landing pages?
How do you spy?
Spying is a great way to learn.
I often swipe banners and landing pages to see how they work, and then modify them myself (change pictures/text etc).
or simply see what kind of offers are running ,,,
Yep, I generally find spying far more useful as a research tool than anything else.
A couple of things spring to mind that haven't been mentioned yet:
1) Spied landing pages, ads, etc are great as a way to get a baseline. Never run a particular vertical on a particular traffic source? Running a few ads or pages closely based on spied content will let you get an idea of the baseline that other successful ads can expect: from there, you have a basis to measure your original ads, angles, landers and so forth against. As Ruby Tunes says, you can then modify them aiming to beat that baseline.
2) Spying on traffic sources that are similar to the ones you're currently using, but not the same, is a great way to get fresh ideas for your traffic source. Hardly anyone does this, and it's really powerful. For example, back in the day when FB was heavy on dating, you could spy on POF, note hot angles that were working there that hadn't been tried on FB yet, port them over, and they'd often work just as well on that new traffic source...