The citation problem

When non-classicists write about Homer, it seems they’re allergic to reading any actual research on Homer. This can be a problem.

Earlier this month there was a lot of press coverage for an article which analysed social networks depicted in the Odyssey. The piece was written by three physicists, and came out in PLoS ONE, a major open-access science journal. The idea had potential: it could have told us some interesting things about how late Iron Age people imagined social relations. Unfortunately, the idea they chose to put front and centre is a ridiculous claim, and it undermines the whole project. In their own words:

How we showed Homer’s Odyssey is not pure fiction, with a little help from Facebook.

-- The Conversation, 3 Sep. 2018

That’s nonsense, of course, but I want to look at a more fundamental problem. What research did they do? Here’s a diagram of how the citations in their article look.

The outer ring represents the 48 sources they used, the inner ring represents the citations of those sources. Click on the image for a closer look. The exact figures are:

• 2 sources on ancient history, cited a total of four times (light red)
• 37 sources on network analysis, cited eighty times (yellow)
• 4 sources in other fields, cited seven times (green)
• 3 sources aimed at general readers, cited nine times (light green)
• 2 pieces of software documentation, cited once each (darker red)

So this is supposedly an article analysing the Odyssey, right? Yet at no point did it occur to them to see what research anyone else has ever done on the Odyssey. It didn’t occur to the PLoS ONE editor, either, a psychologist at Austin. And it didn’t occur to the referees who did the peer review.

For reference, the sources they cite that do relate directly to antiquity are:

• ‘ancient lit/history’: actually more archaeology than history. One is a 2003 piece on the geology of the Troad; the other is from a generalist magazine, not a piece of research, but I’ve chosen to put it here because it’s by Korfmann.
• ‘pop translations and textbooks’: two popular translations of the Odyssey (Rieu 1946, Palmeira and Correia 1944); and a pedagogical companion by Peter Jones.

The references generally are a mess. They misspell Korfmann’s name, they attribute Peter Jones’ book to a different author (it’d be very hard for a non-specialist to work out what book it is!), and several of the DOI links in the article’s references are broken (references 11, 12, 20, 21, 41, and 48 -- including both of the archaeology articles).

Let me re-state the problem. It didn’t occur to anyone, at any stage, that a research paper ought to look at research on the thing that the article is about. Why not?

It isn’t an isolated occurrence. Here are a few more, by various scientists: a 2008 piece by two astronomers, supposedly showing that the Odyssey refers to a solar eclipse that took place in 1178 BCE; a 2012 piece by another astronomer, about another eclipse; three chapters in a 2008 book, written by an engineer.

Now, the first two are much more variegated than the 2018 social networks study. The lion’s share of references still go to things in the authors’ own fields: in itself that’s fair enough. And, though I haven’t included it in the pie-charts, they both have a goodly number of references to the text of Homer (unlike Miranda et al., who have none).

But there are very few references to modern research on the subject of the articles -- namely, the Homeric epics. Where they do cite research pieces, they’re handled strangely. In the case of Baikouzis-Magnasco, they’re badly chosen; in Papamarinopoulos et al., they’re narrow and misleading. In the Baikouzis-Magnasco article, the only pieces cited are

• Denys Page, The Homeric Odyssey (1955)
• Gilbert Murray, The Rise of the Greek Epic (1924)
• Robert Bittlestone, Odysseus Unbound (2005)

Page and Murray are old, Bittlestone is kinda fringe. They’re cited once each. At one point the authors actually mention consulting footnotes in translations of the Odyssey as if that’s what research looks like.

Never mind the authors, is that really what referees and journal editors think a literature review looks like?

Papamarinopoulos et al. actually cite Russo’s commentary on Odyssey books 17-20 at one point -- but without a page number, and Russo doesn’t say what they claim he says. (The claim is that Od. 19.306 λυκάβας means ‘the time period between old and new moon’: that’s not Russo, it’s a misreading of Od. 19.307.)

This isn’t just a scientist problem: it isn’t an arrogant I-know-a-lot-about-my-field-therefore-I-assume-I-understand-everything situation. Or if it is, it’s a symptom of something deeper. A related phenomenon also appears in the Wikipedia articles on the Odyssey and Iliad:

Now, the Wikipedia articles do in fact cite academic sources a lot more than the research articles above. And that’s good. However, we’ve also got loads of references to dodgy websites and news media. And that’s bad.

Also, though the ‘Odyssey’ article has lots of citations of modern research, there are only two sources. 85% of them (29 out of 34) are citations of Agathe Thornton’s book People and Themes in Homer’s Odyssey (1970). Now, I love Thornton’s book ... but that’s just not a balanced treatment.

I have two hypotheses as to the root of this problem. The first is the one you’ll probably be expecting. Homeric research is difficult to get to grips with, there’s a hell of a lot of it, articles often don’t translate the Greek. Also, many of the highest-profile books spend ages harping on about the Homeric Question in one form or another, and no one wants to read that.

But any field is complex and difficult for an outsider. I have a tough time working out what’s going on in the more mathematical parts of articles on archaeoastronomy or stylometry, but I still read them. The authors above didn’t even look at any research. So I don’t think this can be the main reason.

My second hypothesis is that the existing research is actually invisible to them. This applies to people in the sciences in particular. It’s because there’s very little overlap between bibliographic databases that cover the natural sciences, and bibliographic databases that cover Homer or other topics to do with antiquity.

Let me illustrate. Open up a new browser tab and go to a nice general database: Google Scholar. Type in a search for ‘Odyssey’. What do you see?

Your results may be different from mine, so I’ll tell you what I found. The first page of results had only two results that were relevant to the Odyssey -- and they’re both references that appeared in the scientific articles above: Page’s The Homeric Odyssey, and the Oxford Commentary on Homer’s Odyssey. I suspect that isn’t a coincidence. There was one more result on page 2, one on page 3, four on page 4, two on page 5, and none at all on page 6.

Guess how far I had to go before I found any research articles.

I finally found the first journal article at the bottom of page 7. The 68th result. It was Helene Foley’s classic piece ‘Reverse similes and sex roles in the Odyssey’.

Published in 1978.

For reference, Homeric scholarship has 200 to 300 publications a year, as reported in the bibliographic database L’année philologique. Since 1978 there have been a bit over 9000 publications.

But every single one of them is invisible, because Google Scholar doesn’t know how to interpret the word ‘Odyssey’. It’s hopeless on ‘Homer’, too: I couldn’t see a single relevant reference in the first fifty pages of results. ‘Iliad’ is better, but still very book-heavy: there’s only one article in the first four pages of results (Willcock’s 1964 paper on ‘Mythological paradeigma’).

Obviously, a partial workaround would be to search more intelligently. Searching for ‘Homer Odyssey’ returns relevant results. But that’s not going to catch all situations: the authors above could have searched better, yes, but the blame isn’t solely on them.

Basically, unless Google Scholar decides to improve its algorithms, you can expect to see more scientific papers on Homer, written by people who’ve done no research on Homer.

In closing it’s only fair to look at what citation practices I’d recommend. Here are two more pieces written by scientists -- but what a difference!

This isn’t an endorsement of their arguments, by the way: they’re both deeply flawed articles. But they do handle their literature reviews responsibly. (Well, kind of: Altschuler et al. only have 14 references.)

And here are some illustrations from within the field: Foley’s 1978 article on ‘reverse similes’, and a 2012 piece I wrote in response to the Baikouzis-Magnasco article.

Also, take a look at what Wikipedia does with the ‘Homer’ article. This is a totally different kettle of fish from the ‘Iliad’ and ‘Odyssey’ articles. Behold, and admire. If you actually read it, it’s still obvious that it’s not professional -- but the research principles are not half bad.

Will Google Scholar and other similar search engines step up to the challenge? I don’t know. Right now, things aren’t looking promising.

References

• Altschuler, E. L., et al. 2013. ‘Linguistic evidence supports date for Homeric epics.’ BioEssays 35.5: 417-420.
• Baikouzis, C.; Magnasco, M. O. 2008. ‘Is an eclipse described in the Odyssey?’ Proceedings of the National Academy of Sciences 105: 8823-28.
• Foley, H. P. 1978. ‘“Reverse similes” and sex roles in the Odyssey.’ Arethusa 11.1-2: 7-26.
• Gainsford, P. 2012. ‘Odyssey 20.356-57 and the eclipse of 1178 B.C.E.: a response to Baikouzis and Magnasco.’ Transactions of the American Philological Association 142: 1-22.
• Miranda, P. J.; Silva Baptista, M.; de Souza Pinto, S. E. 2018. ‘The Odyssey’s mythological network.’ PLoS ONE 13(7): e0200703.
• Paipetis, S. A. (ed.) 2008. Science and Technology in Homeric Epics. Springer.
• Panchenko, D. 1994. ‘Thales’s prediction of a solar eclipse.’ Journal for the History of Astronomy 25.4: 275-288.
• Papamarinopoulos, St. P., et al. 2012. ‘A new astronomical dating of Odysseus’ return to Ithaca.’ Mediterranean Archaeology and Archaeometry 12.1: 117-128.
• Wikipedia 2018. ‘Homer.’ Retrieved 18 Sep. 2018.
• Wikipedia 2018. ‘Iliad.’ Retrieved 18 Sep. 2018.
• Wikipedia 2018. ‘Odyssey.’ Retrieved 18 Sep. 2018.

The citation data was prepared with this spreadsheet (LibreOffice format). See the note at bottom of the first sheet on some differences in how citations are reported on different sheets.