Which are the top development journals?

In my research, I’m examining development journals. And I wanted to select the top journals in the field – in academic terms, that is – because I think this is where the publish or perish pressures will be highest.

In order to select a sample of the journals, I’ve considered three different systems for ranking journals: the ISI Web of Science (WoS) Impact Factors; the categories developed by the European Association of Development Research and Training Institutes (EADI) and the Dutch research school, CERES; and a ranking system developed by Richard Heeks of the University of Manchester, UK.

The ISI WoS ranks all academic journals. Although it is important to consider the ISI citation index, the WoS has no useful ‘development’ category to make it possible to identify relevant journals. The CERES/EADI ranking also includes all Social Sciences journals, not a sample of key journals. Indeed, there is considerable inconsistency in the different ranking systems. For example, only 3 of the journals in Heeks’ list of the top 24 journals has an A rating in the EADI/CERES ranking. Which is pretty strange, dont you think?

The Heeks’ approach has the advantage in that it comprises a list of core development journals based on a comparison of different ranking systems and a transparent selection process:

–  Selection was on the basis of development studies journals that appear in various other tables or lists.

–  Citation score was calculated by taking papers published in each journal in 2008 and identifying how many times each paper is cited in Google Scholar.  The average number of cites per paper was then divided by the average number of years since publication.  The score, therefore, generally equates to average number of Google Scholar citations per paper per year.

–  All papers published in 2008 were used if less than 20 were published; a sample of at least 20 building outwards from the mid-year issues was used if more than 20 were published.

I’ve used the Heeks’ list as the basis of the selection of the top journals because he has made a shortlist of the top development journals but then include only those journals which currently have an ISI ranking, which have an A/B ranking from EADI/CERES, and which have a development focus. This gives a list of 8 journals comprising:

Journal  Heeks’ list  ISI

citation score

CERES/EADI
Rank Citation Score
World Development 1 6.04 1.612 A
Journal of Development Studies 2 4.90 0.793 B
Development Policy Review 4 3.20 0.869 B
Sustainable Development 6 2.39 1.209 B
Development and Change  8 1.89 1.359 B
Journal of International Development 11 1.46 0.793 (taken from website for 2010) B
Public Administration and Development 14 1.21 0.783 B

I decided to exclude ‘Studies in Comparative International Development’, also selected using this criteria, because I think that it does not qualify as a development journal because it has a world-wide focus.

In conclusion, this list of 7 journals is probably where the greatest ‘publish or perish’ pressure is to be found. Burgess and Shaw (2010) in their 2010 article Editorial Board Membership of Management and Business Journals: A Social Network Analysis Study of the Financial Times 40 also made a selection of the top journals in the management field and they were able to justify their choice for the same reasons. We have excluded development economics journals from this sample because of their different citation pattern, dentified by Heeks, but also because the list currently includes the key, academic interdisciplinary journals.


Advertisements
Posted in academic journals, development studies, ranking | Tagged , , | Leave a comment

Questionnaire surveys 1: some common pitfalls

When I undertook a research project of Dgroups while I was working at KIT in 2006-2007, I sent out questionnaire to thousands – literally – Dgroup facilitators in three languages (English, French and Spanish). This was quite frightening but it was a baptism of fire as far as questionnaires are concerned.

I was using Surveymonkey which is a great tool and it allows you to send and analyse many questionnaires – if you have the paid account, that is. Since then, I have been helping colleagues and some fellow students with their questionnaire surveys and I thought it would be fun to share some common pitfalls, relevant for practitioners and researchers.

  1. Dont do it just because you can
    Online surveys – like Surveymonkey- allow you to send out questionnaires to huge numbers of people. But you still need to be realistic and not send out questionnaires indiscriminately because, not only are you reducing the chance that anyone will ever answer your next questionnaire, you are also spoiling things for others. Just because you can send a questionnaire, doesn’t mean that you should.
  2. Make the punishment fit the crime
    If you are using the questionnaire to evaluate a day-long workshop, for example, only use a small number of questions. So often, I have seen that colleagues have the tendency to ask endless questions – because they can – which can take up to 30 minutes to fill in but this is totally out of proportion with the workshop – and will mean that not many respondents will be inclined to answer. As a rule of thumb, if you are evaluating a day-long activity, don’t ask more than 10 fairly simple questions. This means that you need to think about what you really need to know… Always a good thing anyway.
  3. Be careful with forced choices
    It’s important to be very, very careful with forced choices in questionnaire questions – because this is one of the moments when your respondent will leave the questionnaire and never return. This always really irritates me, particularly when it I am forced to choose between answers which are either not applicable or to which I don’t know the answer.
  4. Avoid obligatory answers
    You should keep obligatory answers to the minimum. Your respondents will answer relevant, clear questions so there is no need to make them obligatory. Only use this option when you really have to have an answer to a particular question. For example, something like country of residence might be key to the results. If you make all answers obligatory, respondents will give up when a question is not totally clear or when they don’t know the answer. Giving respondents the option to skip a question means that they will forgive you an unclear or badly formulated question. Which takes me on to my next point…
  5. Don’t know or no applicable
    When you are asking respondents to choose between different possibilities, you need to give them the option – and it can be a combined option – to say that something is not applicable or that they don’t know the answer. This means that you will know why they didn’t reply to any of the options that you offered. Otherwise, you might think that they missed the question by accident.
Posted in pitfalls, questionnaires, surveys | Tagged , , , | Leave a comment

Epistemic reflexivity

Reflexivity has become, in other words, a hegemonic value of the social scientific field and a weapon in struggles over status and resources within the intellectual field (Lynch, 2000).

Just reading a fascinating article by Karl Maton working at the University of Cambridge: Maton, K. (2003) Reflexivity, relationism and research: Pierre Bourdieu and the epistemic conditions of social scientific knowledge, Space & Culture, 6(1): 52-65. What’s it about?

Summary
Pierre Bourdieu’s “epistemic reflexivity” is the cornerstone of his intellectual enterprise, underpinning his claims to provide distinctive and scientific knowledge of the social world. This article considers what this notion offers for research and how it needs to be developed further to underpin progress in social science. Many reflexive research practices are sociological, individualistic, and narcissistic, and the article contrasts this to Bourdieu’s conception of reflexivity as epistemological, collective, and objective. The author then illustrates how, despite Bourdieu’s intentions, this conception when enacted tends toward the very pitfalls it is intended to avoid. Building on a developing conceptualization of the relations of knowledge, the author identifies this problem as intrinsic to Bourdieu’s framework, showing how it bypasses the significance of knowledge structures and so provides the social but not the epistemological conditions for social scientific knowledge. Bourdieu’s reflexivity objectifies objectification but needs development to help achieve objective knowledge. The article concludes by introducing the notion of “epistemic capital” as a first step toward developing a properly epistemic reflexivity and so realizing the potential of Bourdieu’s enterprise.

My notes
The author talks about different forms of reflexivity. He highlights the role of what he calls enacted reflexivity in the social sciences – which sounds fairly fake to me – which includes autobiographical reflection, the reflective practitioners, hermeneutic narcissism and authorship denial. I liked his description of authorship denial:

Here, reflexivity is enacted as a game of hide-and-seek; one may in effect play “hunt the author” amid the textual play of voices (or, where these voices are all the author’s own, one may have difficulty ascertaining the author’s position). Both forms [this also refers to hermeneutic narcissism] thereby begin by recognising objectification but end by denying it, either through self-absorption or self-denial. (p. 55)

He argues that all these sorts of reflexivity are sociological, individual and narcissistic:

In short, these types of reflexivity in research represent what one could call reflectivity (Turner, 1981;Woolgar, 1988). (p. 56)

Reflexivity has typically been proclaimed as critical and progressive. However, by reducing reflexivity to individualised reflection, the above research practices represent strategies for maximising symbolic capital within the intellectual field at minimal cost. They emphasise individual status (particularly when allied to claims about the “unreflexive” nature of past work in the field) without disturbing the social position and structure of the field as a whole. Ironically, such practices are thereby more oriented toward conserving the status quo than their frequently professed “critical” appellations might suggest. (p. 56)

In Bourdieu’s critique of the academic field, he argues that actors try to impose their own viewpoint on others in struggles for status and resources (Bourdieu 1988a, p. 26). Maton asks how it is possible to ‘overcome the gravitational effects of the intellectual field’ (p. 57) by developing analysis which represents more than the means of acquisition of economic and symbolic capital. For Bourdieu, epistemic reflexivity is the way to do this because it underwrites rather than undermines scientific knowledge. What is epistemic reflexivity? According to Thomas Ryan in the paper When you reflect are you also being reflexive? (undated):

… the constant analysis of your own lived experience as well as your own theoretical and methodological presuppositions (Coghlan and Brannick 2005, p. 62)

Three relations of knowledge
Maton argues that Bourdieu’s distinctive contribution can be breaking knowledge down into three relations (see Fig. 1):

Bourdieu’s main innovation was in looking at the ‘objectifying relation.’ Objectifying objectification underpins reflexivity that is collective and non-narcissistic. Bourdieu (1999, 2000) argues that there are three potential biases in knowledge claims: social origins of the researcher; the researcher’s position in the intellectual field; and viewing the world as a spectacle. But it is not the individual researcher who is of interest to Bourdieu but rather the intellectual field and revealing the ‘collective scientific unconscious’ (p. 58). Maton argues that the intellectual field is not just about maximizing capital that there are other issues at stake, namely: the recognition of the role of non-social interest in producing knowledge which Maton calls ‘the will to truth’ or ‘cognitive interests’  (p. 61) and the recognition of the importance of epistemic capital, or the ability to better understand the social world, aiming for epistemic profits.

2003Space_Culture (Maton 2003)

Posted in Uncategorized | Tagged , , , , , , , , | Leave a comment