On reference letters in academic hiring
It is standard practice to ask applicants for academic jobs – at least for jobs in philosophy – to submit reference letters. Yet, an increasing number of people have been recently expressing scepticism about the practice. (See, for instance, the comments to this blog post.) This may be a good time for public discussions about it; the present post is a sketch of the pros and cons of using letters of reference in the process of selecting people for academic jobs in philosophy.
One worry with hiring on the basis of reference letters is that this tends to reinforce the regrettable importance of ‘pedigree’ – that is, of the university where candidates got their doctoral degree and of the people who recommend the candidate. There are relatively few permanent jobs in the current job market, and significantly more qualified candidates than jobs, whether permanent and temporary. One consequence is that there are many (over-)qualified candidates for (almost?) each job, and this makes the selection process really difficult. Considering dozens, sometimes hundreds, of applications for one position is onerous, so it’s appealing to take pedigree into consideration because this is an expedient method to decide whom to short-list or even whom to hire. (As I mention below, this doesn’t mean expedience is the only appeal of relying on pedigree.) But this is unfair: those candidates who weren’t supervised by influential letter-writers, or who otherwise didn’t make their work sufficiently know to an influential letter-writer, have fewer chances on the job market. Moreover, relying on letters of reference can also be bad for quality, to the extent to which letters fail to closely track merit. This kind of problem will not entirely go away just by eliminating reference letters – the prestige of a candidate’s university will continue to matter – but its dimensions would be more modest.
Another worry is that reference letters reflect and perpetuate biases, perhaps unconscious ones, against members of groups that are under-represented in philosophy for reasons of historical or present discrimination, such as women or racial minorities. There are studies suggesting that reference letters written for female candidates tend to make recommendations in terms less likely to ensure success than those used in letters recommending male candidates. If this is true, letters of reference can, again, be unfair.
Another group of reasons to give up reference letters has to do with avoiding corruption – of the hiring process and of personal relationships within the academia – and their unhappy consequences. As long as they are used in hiring, reference letters are highly valuable assets. Those able to write the most influential letters can treat letters as tokens in exchange for illegitimate benefits from the candidates they recommend. Hiring without reference letters would diminish the potential of unfairness towards candidates who resist such exchanges, and the likely unfairness towards others when they don’t. At the same time, it would eliminate one source of illegitimate power of letter writers over the candidates they recommend. To depend on a supervisor, or on other powerful people in the field, for a reference letter, seems in itself undesirable: a relationship in which one person has enormous power to make or break another’s career looks like a relationship of domination. But even if it isn’t, there might be a good reason against such dependency: to protect the possibility of genuine friendships between established and budding academics. Genuine friendships are more difficult to flourish when structural conditions make it likely that people who pursue the relationships have ulterior motives. It is great not to have to worry that your professor cultivates your company because they want from you something they could ask in exchange for a reference letter. It is great not to worry that your student or younger colleague cultivates your company because they hope you’ll give them a good letter. (Not all such worries will be averted by giving up on reference letters; one can and some do advance their students’ careers by unduly facilitating for them publications in prestigious venues or by over-promoting them.)
Finally, there are reasons of general utility to hire without reference letters: Writing them takes time and, unlike other writings, they rarely have value beyond getting someone hired. (Plus, it’s not the most exciting way to spend one’s time). Interpreting reference letters properly can also be a drag in the context of praise inflation. And it is stressful to ask people to write letters recommending you. Admittedly, these are not, in themselves, very strong arguments, but they should nevertheless count in an overall cost-benefit analysis of reference letters.
All this is not to say that there are no advantages of having reference letters in the hiring process. They may be useful proxies to determine that a candidate received proper philosophical training: in philosophy, at least, we learn an awful lot by simply seeing how others do good philosophy, and being allowed to participate in the process. The mere fact that one has been taught by a respected philosopher should count for something. But, in fact, in this day and age it is becoming increasingly easy to witness good philosophy independent from who mentors you. There are numerous conferences, and it became easier to travel to them; the internet is turning into an inexhaustible source of filmed philosophical events. Almost everybody can study the best philosophers in action, and many can interact with them on a regular basis at philosophical events. These philosophers will continue to feel particularly responsible towards their own students’ careers (and so write letters for them) but, thanks to contemporary media, they can benefit an ever higher number of students in philosophy. Of course, search committees will not know which candidates who were not taught by the most prestigious philosophers did in fact benefit from the easily available resources (conferences, recordings of lectures and other events.) But nor can they assume a wide gulf between candidates who have and candidates who have not been taught by very respected philosophers.
A very important function of reference letters is, to my mind, that of giving prospective employers a way to check certain aspects concerning candidates, in case of doubt. This pro is specific to references, and as such has nothing to do with pedigree. Is a prospective employee a good team player? How much did she contribute to a particular publication? How close to publication are the papers listed on a c.v. as ‘in progress’? But this aim can be satisfied in the absence of a general requirement that job applications include letters. It is enough if candidates are asked to list, in their application, a few people who can comment on them, with the understanding that referees are only to be contacted occasionally/exceptionally.
It appears that, on the balance of reasons, the pros count less than the cons. Letters may be important when employing people in many other fields because, together with interviews, they form the main basis for assessing a candidate’s ability. But in philosophy hirings, where both written samples and job talks are required, we could and probably should do without them.
Anca, thanks for this post. One additional argument against letters could be that it conceptualizes individual who are (or should be?) „grown up“ academics as disciples of others. One might discuss about where letters of reference should „stop“, e.g. whether they should be used for graduate students, but not people with PhD etc. But that’s a minor point.
One counterargument against your position, however, is that letters of references formalize mechanisms that are there anyway. If you don’t have letters of reference, there will be the phone call from the former supervisor to the head of he hiring committee. So some would say: better have an official system, in which all applicants submit letters of reference, and get rid of these informal influences. I am not sure it is a good argument, because I am not sure whether having letters of reference actually puts an end to the phone calls. But its noteworthy that it is based on benevolent intentions, at least on the face of it. Do you think it is worth taking seriously?
Thanks for the post Anca.
I want to develop upon something you mentioned. Having had an academic position in five different countries, I think it is also worth noting that interpreting these letters is made very difficult because of different cultural standards and practices. In Belgium, I was surprised that the student writes the letter themselves and the prof. simply edits points of contention and/or out any hyperbolic language. In other countries, such as Canada, I sometimes felt like professors were competing to find the most hyperbolic language possible and everyone had to fall in the top 5%. Another cultural problem is that some staff members don't feel comfortable writing letters in another language nor do they appreciate the style of letters they receive from profs in other countries because of their own bias against 'arrogant' language. In the end many of these cultural questions might harm more than help a candidate.
Thank you Anya.This is funny: Belgium, too. In Romania when I was a student it was standard that students were writing their own letters. Yes, I think this is a very general point: both the usefulness of letters, and the costs of having them in the process, increase when you take a more global perspective.
Thank you Lisa. I am thinking about the first point. I know in some countries the association with 'being X's disciple' is very strong; I hope it is not a necessary one.
But I agree with your second point, that ditching references will not go hugely far. I really don't know whether people would put in place other, less formal, mechanisms to exert influence or accept to be hands off.
I think here it is worth noticing that many other hiring processes manage to do without references. A relative of mine works as administrator in a UK university, and has often expressed surprise at the role references play in hiring academics in the UK. When she has been involved in hiring other administrators, they look at references after having made a hiring decision, primarily to check that the candidate hasn't been lying about their employment history. So far as I can see, they're a way of perpetuating an old boys network and ought to be straightforwardly resisted as such. It's not just that they put presently senior people in a position to exploit presently junior people for various kinds of favours – and remember that this isn't just a matter of writing letters praising them to the skies, but of refusing to write letters or of write bad ones – but that they are able to exercise an unwarranted power over the shape of the discipline in much the way that Lisa Herzog suggests by encourage their colleagues to hire people whose work they like.
Yes, great post Anca. I suppose if we ditch references then it should become clearer that the phone calls are out of order. After all now one can rationalise a phone call (or email) by saying that it's just an extension of the reference system.
Let me also add one general comment. It's good to curtail the mechanisms through which the old boys network operates. But inequality of outcome is the proverbial ruck in the carpet. We obsess about trying to equalise opportunities, but that largely just moves the ruck around. You mention informal promotion of candidates and facilitation of prestigious publications. Well, my fear is that if we eliminate references we'll see more of those practices. What we need is an academic environment where big shots just can't get that big — an environment with more uniform working conditions across institutions, less snobbery, the works. All this applies differently to different countries, with the Anglophones the worst offenders, but as academia becomes more international the Anglo model tends to dominate, whereas I think there are a lot of positives (and a fair few negatives) in the relatively egalitarian Northern European academic model.
I agree Enzo. The longer I think about this, the less concerned I am with equal *opportunities* rather than equality full stop (for reasons that go well beyond the content of this post.) Yet, doing without references is something that individual academics, or rather departments, can decide on their own. Whereas changing the entire (higher) education system should, I suppose, come from top down.
Thanks Anca. With regards to Lisa's first point, I suppose most of the objections your post and the comments below raise against the use of reference letters for academic jobs also apply to the use of reference letters for graduate school/PHD applications. It might be that there is a stronger case for using them to support such applications: the applicants usually have no publication record, writing samples or undergraduate records might not necessarily reflect an applicant's ability to carry out graduate work, etc..
Nevertheless, we could think of alternative methods to reference letters as they are now that provide the needed information and circumvent some of the problems you and others have mentioned. For instance, to address Anya's point we can replace the letter format with a standard "score card". Another option, would be not to reveal the referee's identity to the admission committee, this way it is not the identity of the referee that counts but what they say and this might help with diminishing the power of big names. Do you have any thoughts on this?
Thanks Anca. I was thinking about your proposal that reference letters should not be a general requirement, and instead applications should just list a few people who can comment on the application if needed, as a sort of check-list that everything is in order. (Jonathan Wolff suggests something similar).
I think this would definitely be an improvement. But I worry that the list would result in similar unfairness. There would still be an incentive to list prominent philosophers who can vouch for you to impress the hiring committee.
Perhaps this could be addressed if that section of the application was kept separate and the hiring committee were only able to see the list once they have made their initial selection?
I agree with these suggestions, but let me add one more cautious note. If the hiring systems focus more on other elements, for example writing samples, publications, conference participation, etc., this might shift the ways in which networks work towards these things. Often junior people depend on help from senior people to acquire these CV points. Now, the advantage might still be that we have blind reviews for some of these things. But then you might get help for "polishing" a paper, etc. While I think we should make all efforts possible to exclude unfair mechanisms, the basic fact remains that there is a huge oversupply for jobs, spaces for prestigious publications, etc., in addition to the hierarchies in the system that Enzo mentioned. Together, these two facts can create rather unhealthy imbalances of power. But I am not sure what can be done about it – discourage people to do philosophy? Really? Maybe the best thing that could be done is strengthening the exit options for junior people (for example by having philosophy students do internships in areas that are alternatives to a career in philosophy, offer minors in business management?). There is a lot of existential anxiety in philosophy – if people knew that they could also find (good, interesting, challenging) jobs outside of academia, all this pressure might be reduced. We are talking about highly educated individuals in relatively rich countries, after all!
This is a brief reply to worries expressed by Enzo, Lisa and Bruno above. I don't think we can eliminate from academic philosophy all the unfairness that flows from individuals' free association. Some of this unfairness could only be avoided if we prevented people from talking to each other, really. Because who talks to you, and how much attention they pay to you, will make a huge difference to the quality of the stuff you produce as a philosopher.
To my mind this is a pro tanto reason to go for more equality of status, financial rewards and other benefits among academics. (As Enzo suggests, too.) Another conclusion is that some of the biggest worries about reference letters stem not from fairness, but from avoiding corruption and from efficiency.
Could you say a bit more about why you think the influence of pedigree would be more modest if letters were not supplied? Philosophy is a small enough world that hiring committees know (or could easily enough look up) who the Philosophers of AOSX are at Fancypants U of Origin. If the name brand of the letter writer (and not the hyperbolic language, which many letter writers share) is doing the work, committees will still know what names trained the candidate. So I don't see yet why university affiliation wouldn't just take over for letters in terms of pedigree signaling to whatever degree the hiring committee is currently influenced by pedigree.
That said, here's one advantage to dropping letters you didn't consider: in some cases students choose a more famous over a more engaged or more able advisor in order to secure what they take to be all-important letters or pedigree. If there were no letters (and, I suppose, dissertation advisor weren't standardly listed on the CV) then there would be less pressure to make that kind of choice. So in at least some cases young philosophers might get better training.
Lisa, here's a *very* new webpage listing recent hires of philosophers outside the academia – very timely:
http://dailynous.com/2014/05/21/philosophers-hired-outside-of-academia/
Thanks for the second point, I think it's correct.
About the first point: I assume that the content of the letter most often does carry some importance, after discounting any hyperbolic language – at least in the case of better letter writers, who take the time to explain *why* they think a candidate is good. Of course, the more informative a letter is, the higher the cost (for hiring committees) of doing without it.
I have recently been involved in a search to fill an assistant professorship at a continental European university. There letters of recommendation are not considered an essential component of a job application, and we decided not to use them. We actually instructed candidates not to have them sent. In the searches I have participated in in the US, I had always thought that the letters were useful inasmuch as the letter writers could often zero-in on the most important points of a candidate's work in a way that the candidates themselves could not (failure to see the forest for the trees). But I have to say that in this European search I did not miss them. Many of our top candidates had been out for a while and had significant publication records. In evaluating newly minted PhD's the letters might be more important, especially in addressing questions about the originality of the candidate's work. I could not possibly evaluate applications for graduate study without the help of the letters of recommendation.
I just wrote a long response to this which I managed to lose, but here's a shortened version of it:
I agree with Anca and Enzo that a lot of this is about the limitations of an equal-opportunity focus in a context of inequality of condition or outcome. But since the main inequality at stake is between having or not having an academic career, the kind of changes we need for equality involve changes in occupational structure and not just greater equality among academics.
Anca puts forward some very strong arguments, but I have a few reservations.
First, references allow your referee to brag for you, rather than your having to do all your own bragging on your own behalf.
Relatedly, it is a bit more credible for a third party to say you are great than for you to say so yourself.
Thirdly, since referees are putting their own credibility on the line, one has to work on the presumption that they really believe what they say. What they say can be helpful in a situation of making a momentous decision on the basis of very little evidence.
Fourthly, since (as others have pointed out) any bias that an appointment committee might carry about big names or prestigious departments is likely to find its way into the process with or without references, it might be better to tackle that directly, e.g. by addressing it in the codes and training that more and more institutions use in hiring processes.
Finally, references provide a relatively rare opportunity for people to learn that others really value their virtues and abilities. On the (numerous) occasions on which I've failed to get a job or promotion, I've taken consolation in the fact that at least I had some positive references. I hope that the references I've written have given similar consolation to others.
It was all much more eloquent in the last draft, but this is much succinct!
I am making a proposal and can be talked out of it.
I would prefer just having quotas and subsidies designed to rectify biases. We have a very hard time proving biases in particular instances. There are cases that are clear but they do not number up to the inequality shown in the results. Cases that are verifiable are less frequent and verifiable admit contested accounts is even less.
It is good that philosophers are looking at alternative ways to go. But it won't be much easier if some departments change while others do not.
An easier way to go may be to ask for a letter describing the role women philosophers have played in research. If the letter hems and haws, or righteously objects, you can set it aside.
If a journal or a large conference published or scheduled no women, they now know that they are part of an emergency in the discipline. They should then set aside positions for their next issue or sessions. Then they should invite a woman to a position that chooses speakers or papers. Repeat until we are less embarrassing.
Different journals should decide on the number that makes a reasonable quota. %50 is too high given that many women, for good reasons, are working with mostly-women organizations within philosophy or they pursue interdisciplinary work, also for good reasons. The number should bear some connection to the proportion in employment and as graduate students.
The worry is that this will make it difficult for women to show credentials, given that many would believe their publications and appearances are the product of this affirmative action.
Given how bad things are now, it seems worth a shot.
Here's the rant for today. The letters have a very simple role in the US/UK/Canada academic system (in philosophy), and an essential one: to ensure that you are among the 3-4 people on the short-shortlist for the job. This is so because of the current hiring practice: people get jobs right after they finished their PhD (or even before). This means that these job seekers usually don't need to / are not supposed to 'prove' themselves outside the cocoon of their own famous dept. Most of them have very limited teaching experience and, crucially, they don't have publications. Thus there are no other independent / more objective means to be evaluated on — except for their PhD dissertation, which most hiring committees will not read anyway beyond the abstract (in most cases b/c they are not specialists; the letters do this job for them too, as a good letter is supposed to be specific). Yes, there is also the in-person in-campus interview, but in most cases one w/o letters doesn't get that far. Thus, the letters from the advisor(s) provide the *only* measure of who these candidates are. Hence the situation decried here: the more famous your advisor is – and more pleased with how you obeyed their 'training' directions (read: promoted their philosophical ideology) – the better the chances to get a job. (Anca, and others, guardedly mentions training as a possible positive — and I still can't believe how ridiculously overrated and under-examined this 'training' is; taming is a much better word — btw, do you know of anybody who was trained by famous philosopher X and employed this marvelous training to write a thesis aiming to ruthlessly criticize X's views?…and also got a research job afterwards?? I have no evidence for this, but it's verifiable, and I'm ready to bet famous X's salary on it!) Speaking of the fear of more objective measures, I heard anecdotes that it's actually better not to have any paper published when going on the job market if coming from famous U of X, b.c this will 'expose' you – no matter how good, no paper is perfect; the quality of the journal may be an issue (if not extremely lucky); the reference list is finite and if your file happens to be evaluated by philosopher Y whose paper you haven't cited then you're in trouble, etc. Le(i)tters are safer and here to stay, they have a well-defined goal: to ensure that the research jobs are open only to those 'trained' by primadonas (regardless of how good these candidates are!), and thus, implicitly (but who cares, frankly), to block the careers of smart, hard-working students w/o famous advisors. Amen.
Agreed (unsurprisingly). Much of the nastiness in the profession stems from its highly unequal distribution of resources. But equality could only ever be equality among those inside the profession. An even bigger problem is the sheer scarcity of professional positions in philosophy. Thinking about the name of this blog, it may be that there can't be justice here if the level of scarcity is not at all moderate. Which brings us to the old chestnut of the oversupply of PhDs. If we need to exclude some people to create something similar to the circumstances of justice, I'd say it's better to exclude them early (pre-PhD), when they are less invested in the discipline and have more options for alternative careers. Now that ties in nicely with the point about inequalities: it's in the interest of the better off academics (and administrators) to have this oversupply of PhDs, but it is to the detriment of equality and justice within the profession. It's also against our interests, since it makes us more expendable. We need to take a few lessons in self-regulation and recruiting from medieval guilds. Oh wait, how did universities start out in Europe? Admittedly we did take a few devilish bargains since then, and now we're paying the price.
There are generous helpings of truth among the hyperbole in the above post (somehow I feel less bound to civility when replying to anonymous people). I don't quite see the reference system as a conspiracy of the primadonnas, though there is an element of that. Here's a rough list of the main drivers of the current letter-writing system, as I see it. Forgive the overlap between items.
– People want to be in the good books of big shots.
– People want to be associated with the glamour of big shots and their Leiterrific departments.
– People create a loose debit/credit system that becomes useful when placing their own PhDs.
– Some of the people with clout also like to shape the discipline in their own image/according to their idiosyncratic preferences.
– Promise is sexier than actual achievements.
– People lump together expertise in producing philosophy with expertise on who would make a good colleague in a specific department.
Those are all bad reasons to use references. They are also the main reasons that drive people, I reckon. As pointed out by a couple of people above, there are also good reasons. But I still agree with Anca that the bad far outweighs the good here.
John, I never thought about most of the things that you say here, thank you! I'm not an expert in what brings success on this market, but don't you think that candidates have to brag for themselves anyway if they want the job? Hoe many can reasonably afford not to? I also wonder how often do candidates get to know the content of the letters sent on their behalf. My own experience is that this is exceptional.
Anca, thanks for the post. I am sympathetic to your position and much of what I might have said has been mentioned above. So, two brief points only to press some details. First, I would add a little to John Baker’s comment about the complementary work that can be done by reference letters. I agree that it might reduce some onus on the candidate to brag. But perhaps more importantly, it might also be the case that there are areas that are more easily covered by reference letters. In applications and interviews, one is expected to talk about what one has taught, teaching methods, and so forth. But it is much more difficult to talk about how one’s teaching was received by students (I guess because it comes across as awkwardly ingratiating), even though this information is, in some ways, as important, if not more so. Reference letters can fill this space.
On a broader level, it seems, as you suggest, that much here is about balance of reasons. To that extent, we also need to think a bit about the weight of difference factors in decision-making processes and possible different weightings in different combinations. To try to stylise an example, consider your point about reference letters and pedigree. No doubt it retains import in the UK, but I imagine it has reduced import now that the current REF regime has led to publications having such a large role in hiring decisions. Similarly, partly as a consequence of this system, often panels include members not from the field, such as VCs, who are, thus, less susceptible to the ‘big name influence’ from that area. None of these points are meant to suggest the letter-pedigree point does not hold (or endorse these alternative structures). But I do wonder if our assessment of the reference system might be usefully placed in a big picture – perhaps with some alternative changes, its negative effects could be reduced and its positives held?
Thanks Andrew! I don't know how often do letter writers know anything about the candidate's teaching that does not rely on student evaluations. (And one reads lots of worries that this is not a reliable source these days.) If student evaluations are the source of knowledge, then A digest can be directly included in the application I guess. Even better: it may be a good idea to ask prospective holders of *permanent* jobs to demonstrate teaching?
I agree with all the worries about the letter system pointed out since now.
Yet I have one doubt especially as regards referenze letters used for admission to PhD programs. My impression is that, without the observations and evaluations of a well-known and competent professor, the name of the department or university where the candidate got their degree will appear much more prominently. This can be harmful for candidates from less prestigious universities and, I suspect, especially for candidates from countries other that US and UK.
Indeed, I've always seen letters commenting the student's work and abilities written by scholars other than the student's own mentor from internationally-unknown-university as a chance to get recognition from an international committee. Obviously, these remarks apply to the present system, where country and university sometimes count more than writing samples and even publications.
Prof. Gauker, I'm curious how many candidates there were for the position you were filling. The reason I ask is this. I have either chaired or served on seven or eight search committees at my current institution. In each search, the consensus on the committee is that the only really important part of a candidate's dossier is the written work. But our searches often attract more than 250 candidates, and we try to get it down to about 30-40 candidates whose written work we read carefully. Once a candidate is on this long list, the written work is by far the dominant feature of the file. We use CV's and letters as a necessary evil to narrow the field down to a manageable size. I honestly believe that, if we had only 30 applicants, then we'd just go straight to the written work, maybe checking CV's later, and maybe even skipping letters altogether. If you have largish fields (> 150 candidates), do you read everyone's work, or do you use CV's to narrow the field?