I wrote a bit of a philosophical rant on why and how you could DIY CGM (social media/consumer generated medi) research and longditudinal analyses.
The key tool within this is to develop good enough search strings ( list of keywords in a boolean expression) and I will get back to a very simple, artithmentically based emprical method which will make you efficient in "cutting to the quick" of any topic or brand-tracker in CGM.
1) follow the path of least resistance: everyone has trodden there!
- If you get your search strings sorted in your preliminary build, then you will follow a route which all the sheep do, and therefore come upon the forums and bloggs which have the highest postings and the highest reads for your topic or brand.
Forums are for the moment, still the best source of consumer insight for many areas. Remember that google/yahoo etc only index about 15% of the web and probably less of total "live" content so far, heavily biased to some forums in some topic areas. This means you will follow the heard, but remember to go and look at the forum indexes once you identify which are the most ranked in google.
2) Once in the place, read the biggest threads:
subscribe to those which have not become off topic rants. Use the forum search functions, advanced if possible, on your topic of interest. ( go back to your snowballing query build - coming soon on a blogg from DF!)
3) Small can be beautifully ugly!
When looking for say, NPI problems, you want to do the opposite: find small threads or recent ones near a launch date for example, and then see if this has propagated out on bloggs, forums and twitter searches. If you see a smattering of discontent, then this can point to a failure in QA or just a bad product.
4) Follow the opinion leaders: the forum matadors.
Often you can find all posts by an author who seems to be an opinion leader, early adopter or influencer and follow your topic using this route. This is often valuable insight, and also you have a weigthing for importance based on them replying in threads.
5) Use "5 bar gate" counting on hits
for various opinions, co-mentions / brand comparisons and opinion polling: often called sentiment rating. Look for ratings for posts and any polls which appear in the area. Also make a note of "reads" on forums and bloggs. All these are useful stats and you are adding a quality filter which computers cant!
6) Too many results! ??
use an nth page technique: however, for a given date range or from the latest listings, read the first few pages to see if the same threads are getting multiple listings, or bloggs are getting extended replies. An nth page sampling strategy plus using quick hit counts onto a sticky note is just as good as a computer doing a census. For 100 pages, an every tenth page would be adequate, or every 5th if the results are often not in true CGM .
Use this also within huge threads- read the first few pages completely, then do every say 5th before moving up to every 10th until the latest 5 or first posted five (depending on rank order by date ascending/descending is default) thoroughly. Then you know the "currency" and "providence" of the thread and can rate it's value.
Follow links to other forums and other CGMs from your top relevant forum postings. In this way you snowball out and can gain a picture of the prominence on the different SM platforms, and the interconnectivity- itself an interesting meta-metric to consider.
For example there may be a youtube video or a discussion on a related article on the BBC or CNN or so on, which means the prominence in "views" is really very high for a brand, product or topic.
8) Issue tracking:
now of course you can subscribe to a thread, but you can also do a bit of detective work: look back over time and follow external links. Expand your search strings to include.
9) Running statistics:
this is a little bit tediouos but can be done using repetitive searches on advanced SE forms over different date periods, or using five bar counts. Once again for very long results lists you can sample nth pages and get an idea of date-distribution.
A short cut is to know results per page / total pages and when you are in threads, the same for posts per page. It can be a little tedious to get to the last page for the count if there is no "skip to last" button.
10) summarise your stats and findings: priorities around the more prominent media spaces, BBC, CNN; the biggest forums while also pointing to those specialist forums which maybe have the highest QUALITY of insight - early adopters, beta testers, industry insiders, and innovators who are taking your product to bits. From your bar counts you can do a rough guide to opinion ratings for products and illustrate this with quotes and whole conversatipns. You may want to make a couple of case study threads on inlfuence by opinion leaders or on how prominent a single thread has actually been and how "linky" and " tweety" a page is. You also want to include the prominent blogs and the prominent Facebook groups, applications and fanclubs. From all this you could produce a very insightful report on a brand, an NPI issue or on a developing consumer need and this report would likely be of high quality compared to the big hit counter type products.