September 26, 2005

I might actually have something interesting to write about...

The first purpose of this entry is to draw attention to my "nerd score"(bottom of sidebar). I just took this test that I linked to from Ms.PhD's blog. I did NOT expect to be a "High-nerd"!! Hmm. I suppose I'm nerdier than I thought. Oh well.

Now, there is something that I'm trying to figure out that actually has something to do with my research. For part of my thesis I'm looking at the macroscopic charcoal records from 3 lakes. My advisor has done this sort of thing many times before, so she knows how do analyze this type of data. I'm reading *the* paper on the CHAPS program (written by P. Bartlein at the University of Oregon), which is what we use to "decompose" the charcoal record into background and peaks. Supposedly this is the only paper published on CHAPS statistics, and there is of course the manual that goes with the program. I am reading this to hopefully become more familiar with the actual statistical calculations that go into the program. But I have found the paper a bit un-helpfull. (By the way, this is the Long et al. 1998 paper from the Canadian Journal of Forest Research).

I think I understand the methods, but I feel there is something missing from the paper. I wish that the authors would explain their statistical methods better. I said that I am reading it to become more familiar with the calculations they use. But they don't specifically talk about their calculations. They describe their statistics with words, and don't ever show any formulas. I think this is a poor way to get your methods across to your readers. I don't think that I could just read this paper and then do the same thing all by myself. I have someone who can explain it to me, and I have people that I can ask questions independently. But I think it is crummy as a methods paper because it doesn't really give you any methods.

My advisor even said, "some people say CHAPS (the program) is a black box, where you just put in your data and it spits out some statistics. " She went on to say that it is not simply a black box, but that "we" know what is going on inside. I don't think this is true. I think someone knows what's going on inside (namely Bartlein, since he wrote the program and he is pretty brilliant with statistics), but the majority of people who use it don't. I have seen a few talks about this type of data, and usually the speaker just says some things that sound nice, so it sounds like they know what they're doing, but if you ask them specific questions about their statistical methods and why they did some things and not others, they usually falter and can't answer. This is a bad way to do science. I feel that any scientist presenting something in a poster or a talk should be able to answer questions about their data! Even funny questions. You shouldn't have to say, "Well, I know that CHAPS does a locally weighted moving average, but I couldn't do it by myself, I don't really know how to do it." I'm not saying that you shouldn't save time by using programs written for your specific situation, I'm just saying that you should still know exactly what is going on, and why.

No comments: