Multilevel leadership researchers have predominantly applied either direct consensus or referent-shift consensus composition models when aggregating individual-level data to a higher level of analysis. Consensus composition assumes there is sufficient within-group agreement with respect to the leadership construct of interest; in the absence of agreement the aggregate leadership construct is untenable. At the same time guidelines to help leadership researchers make decisions regarding data aggregation issues have received little explicit attention. In particular a discussion of how data aggregation decisions can enhance or obscure a study's theoretical contribution – a central focus of this article – has not been addressed thoroughly. Recognizing that empirical generalization depends on the accuracy with which aggregation decisions are applied we revisit the often neglected assumptions associated with the most common agreement statistic used to justify data aggregation — rWG and rWG(J) (James Demaree and Wolf 1984). Thereafter using a dataset published as part of a Leadership Quarterly special issue (Bliese Halverson & Schriesheim 2002) we highlight the potential misuse of rWG and rWG(J) as the sole statistic to justify aggregation to a higher level of analysis. We conclude with prescriptive implications for promoting consistency in the way multilevel leadership research is conducted and reported.