Chat with us, powered by LiveChat Week 3 – Assignment: Identify a Process for Managing and Organizing Information Instructions | Gen Paper
+1(978)310-4246 credencewriters@gmail.com
  

The Journal of Academic Librarianship 43 (2017) 57–66

Contents lists available at ScienceDirect

The Journal of Academic Librarianship

Comparison of the Accuracy of Bibliographical References Generated for
Medical Citation Styles by EndNote, Mendeley, RefWorks and Zotero

Jiří Kratochvíl
Masaryk University Campus Library, Brno, Czech Republic

http://dx.doi.org/10.1016/j.acalib.2016.09.001
0099-1333/© 2016 Elsevier Inc. All rights reserved.

a b s t r a c t

a r t i c l e i n f o

Article history:
Received 26 April 2016
Received in revised form 18 August 2016
Accepted 8 September 2016
Available online 15 September 2016

Bibliographical references to online and printed articles, books, contributions to edited books and web resources
generated by EndNote, Mendeley, RefWorks and Zotero were compared with manually written references ac-
cording to the citation instructions in 15 biomedical journals and the NLM citation style. The fewest mistakes
were detected in references generated by Zotero for 11 journals and the NLM style, while the second fewest num-
ber of mistakes was found in Mendeley. The largest number of mistakes for 9 journals was found in references
generated by EndNote and in the other 4 journals the largest number of mistakes was detected in RefWorks ref-
erences. With regard to the individual types of resources, the lowest number of mistakes was shown by Zotero,
while RefWorks had the greatest number of mistakes. All programs had problems especially with generating the
URL and the date of access in the reference to online documents. It was also found that several mistakes were
caused by technical limitations of the reference managers, while other mistakes originated due to incorrect set-
ting of the citation styles. A comparison showed that Zotero and Mendeley are the most suitable managers.

© 2016 Elsevier Inc. All rights reserved.

KEYWORDS:
Accuracy
Bibliographical references
Citation styles
EndNote
Medical journals
Mendeley
Reference managers
RefWorks
Zotero

INTRODUCTION

Bibliographical references are an integral part of all scientific publi-
cations. However, their authors constantly struggle with generating
them and repeatedly make mistakes in creating them. These mistakes
should not be treated lightly, because they may lead the reader to
doubt the quality of the author’s research; the author might also have
shown the same carelessness in the references in the following parts
of the article or in the research itself (Biebuyck, 1992). In the past,
some authors have been revealed to have mentioned sources in the
list of references which they had not read. Some authors have also
brought over incorrectly formatted bibliographical references from dif-
ferent articles (Cronin, 1982; Garfield, 1990; Sweetland, 1989; Wyles,
2004). As a result, not only the reputation of the authors but also the
reputation of the journals can suffer when poorly produced articles fea-
turing mistakes are published (Spivey & Wilks, 2004). Mainly the cited
authors and the cited journals are affected. Indexing these authors in ci-
tation databases can become more complicated or even impossible due
to such mistakes (Garfield, 1990).

The causes of mistakes in references can vary. Apart from unethical
ways of citing as mentioned, mistakes have also been reported for de-
cades now to have been caused by mere carelessness and the authors’
inconsistent writing (Bahar et al., 2012; De Lacey, Record, & Wade,
1985; Lok, Chan, & Martinson, 2001; Oermann, Cummings, & Wilmes,
2001). The blame lies with the journals themselves, because they do
not contain very detailed citation instructions for authors. They may

also refer to already invalid versions of the citation styles or they do
not give any citation styles at all (De Lacey et al., 1985; Onwuegbuzie,
Hwang, Combs, & Slate, 2012). Another reason for errors is the existence
of too many citation styles where the authors cannot be sure what the
best approach is (Liu, 1993; Moorthy, 1988; Park, Mardis, & Ury, 2011).

Standardisation of reference rules could provide a way out of this sit-
uation (Freimer & Perry, 1986; Garfield, 1990; Mansfield, 1984;
Sweetland, 1989; Terbille, 1990). However, the situation in medical
journals shows the reverse. Despite the Uniform Requirements for Man-
uscripts being in existence for forty years already (ICMJE, 2015; Patrias,
2007), many medical journal editors request that authors follow the
editor’s own citation rules. No wonder that there is a high percentage
of incorrect bibliographical references. For example, in five general sur-
gical journals, such as Annals of Surgery and the British Journal of Sur-
gery, 11% of references published in the July 2004 issues were
incorrect. Three journals dealing with pediatric surgery (e.g. the Journal
of Pediatric Surgery) had 33.7% incorrect references in the first issues of
the year 2001, and the Archives of General Psychiatry and Journal of
Clinical Psychiatry had 38.5% incorrect bibliographical references out
of 420 randomly selected references published in the September 1980
and 1999 issues. Journals in the nursing field, e.g. the Journal of Pediatric
Nursing, had as many as 42.7% incorrect bibliographical references in ar-
ticles published in issues from the period between September 1999 and
February 2000 (Celayir, Sander, & Celayir, 2003; Oermann et al., 2001;
Reddy, Srinivas, Sabanayagam, & Balasubramanian, 2008). In the past,
on average 23.6% incorrect references were found in such prestigious

58 J. Kratochvíl / The Journal of Academic Librarianship 43 (2017) 57–66

journals as the New England Journal of Medicine and The Lancet (De
Lacey et al., 1985). In all these cases the mistakes were usually in au-
thors’ names, the titles of articles and journals, or incorrect information
concerning the year of the journal or the pagination.

Under these circumstances, a growing interest in reference man-
agers is understandable. Reference managers help to administer biblio-
graphical records, text and picture files and above all they assist in
inserting references into the text that are formatted in compliance
with various reference styles (Zhang, 2012). A number of studies com-
paring the functions of the EndNote, Mendeley, RefWorks and Zotero
reference managers have been published recently. However, these stud-
ies, a summary of which in connection with the results of this study can
be found in the Discussion section, focused only on comparing technical
aspects of the reference managers, not on comparing the accuracy of the
bibliographical references generated from within these programs. Only
a few studies have focused on such comparisons (Homol, 2014).

Kessler and Van Ullen (2005) compared 100 references produced in
accordance with the APA style in the EasyBib, EndNote and NoodleBib
programs, and found that the three applications generated 106 mistakes
altogether. EndNote had the fewest mistakes in references to print pub-
lications while NoodleBib revealed the fewest mistakes in references to
electronic publications. Brahmi and Gall (2006) focused their study on
the quality of bibliographical references for 43 most prestigious medical
journals created in EndNote and Reference Manager. They found that
these applications were not able to generate references for 35–47% of
the journals and the references which were generated differed in 33–
43% of the cases from the recommended style. The bibliographical refer-
ences made in the style of the remaining journals contained 33–46% of
differences. Gilmour and Cobus-Kuo (2011) tested the CiteULike,
Mendeley, RefWorks and Zotero managers for the ACS, AMA, APA,
IEEE and Nature citation styles and they discovered that RefWorks
shows the lowest error level in terms of the average number of mis-
takes. Homol (2014) compared the output from the EndNote, Basic,
RefWorks and Zotero applications based on the APA and MLA citation
styles with the references published in the EBSCO Discovery Service.
She found that none of the programs generated faultless bibliographical
references. RefWorks made the fewest mistakes for the APA style and
EndNote Basic made the fewest mistakes for the MLA style.

None of these studies simultaneously compared the output from
EndNote, Mendeley, RefWorks and Zotero, which are presently the ap-
plications most often tested. It is also necessary to verify the quality of
bibliographical references to different types of documents, i.e. not only
journal articles (Homol, 2014). Therefore, the aim of the study is to de-
termine which of the reference managers generates the lowest number
of mistakes for medical journals’ bibliographical references. The focus
on medical journals is due not only to the author’s role in a university
library, namely providing services for the Faculty of Medicine and
their employees in the faculty hospitals, but also this focus was chosen
with the aim of providing a more detailed analysis of the situation for
medical authors and medical librarians. This study will help medical au-
thors to better decide which reference manager to use. This article can
also guide medical librarians when choosing which reference man-
ager(s) to stress in their information literacy classes. Last but not least,
the aim of the article is to encourage librarians to perform further sim-
ilar analyses of the quality of the input from reference managers for
journals from other scientific fields.

METHOD

Between December 2015 and January 2016 the quality of biblio-
graphical references generated for 15 medical journals and the National
Library of Medicine (NLM) citation style by the reference managers
EndNote (version X7.4, Bld 8818), Mendeley (version 1.15.2), RefWorks
(version 4.4.1376) and Zotero (version 4.0.28.10) were compared. The
journals (see Table 1) were chosen in the following way: the ten medi-
cal journals with the highest number of published articles in Journal

Citations Reports in 2014 were added to the ten medical journals with
the highest impact factor in Journal Citations Reports in 2014. These
journals were chosen either because of the need for the medical authors
to publish their articles in the most-referred journals or because the
journals publish a high number of articles and therefore a large number
of authors work with these journals’ reference instructions. From these
20 journals, the following 5 titles were then excluded: CA: A Cancer
Journal for Clinicians, Frontiers in Human Neuroscience, Molecular
Medicine Reports, Oncology Letters and Oncotarget. This was done be-
cause in some citation managers it was not possible to generate the bib-
liographical references for these journals. The NLM style was added to
these 15 journals for comparison. The format and how to adhere to
NLM was agreed on by the editors of medical journals (ICMJE, 2015;
Patrias, 2007). In this way, a list of journals and the NLM reference styles
was created. The authors of medical journals very often follow these
instructions.

Once the choice of the journals and the reference styles was made,
the publications representing the commonly-mentioned types of print
or online resources were chosen. As Homol (2014) pointed out, an anal-
ysis of other sources is needed. Although journal articles are and will
probably continue to be the most frequently cited type of resources, var-
ious studies show that 16.5% of 81,834 references published in ten med-
ical journals were to sources other than an article (Barrett, Helwig, &
Neves, 2016; Delwiche, 2013; Rethlefsen & Aldrich, 2013). Therefore,
references for various types of resources were tested in this study.
These types were a journal article, a contribution to an edited book, a
book, an edited book and a web resource. In addition, in the case of a
journal article, a contribution to an edited book and a book, publications
with different numbers of authors were also chosen. In the case of a
journal article, a contribution to an edited book and a book, publications
were chosen with different numbers of authors so that it could be veri-
fied whether the reference manager is able to generate the number of
authors’ names determined by the journal style. In this way, 17 publica-
tions (Table 2) were chosen. Bibliographical references were manually
created according to the citation instructions of the NLM style and the
example of all 15 journals. The reference instructions were found in
the instructions for authors accessible on the websites of particular
journals. These instructions for authors were the primary source of ex-
amples and were strictly followed. For example, according to the in-
structions of The Lancet a book title was set in capital case and a
journal title in italics, while according to the NLM style a book title
was set in sentence case and a journal title without any changes. If the
instructions did not contain a citation instruction or an example biblio-
graphical reference for some type of resource, a citation of the given re-
source published in the relevant year of the particular journal was used.
For example, reference 37 in the article “Tumor Regression After
Brachytherapy for Choroidal Melanoma” served as an example of a bib-
liographical reference to a contribution to an edited book, because In-
vestigative Ophthalmology & Visual Science does not provide any
example in its citation instructions (ARVO, 2015; Rashid, Heikkonen, &
Kivelä, 2015). In this way, 17 examples of bibliographical references
were made for individual journals and the NLM citation style.

After the examples of bibliographical references were created, man-
ually-created records for the same publications were made in each of
the reference managers. These were made manually so that they
would contain all data in particular fields of records and so that mistakes
due to importing records from different sources could be avoided
(Basak, 2014; Kessler & Van Ullen, 2005). Following this, bibliographical
references were generated from each reference manager using the cita-
tion formats for particular journals. The bibliographical references were
compared with the examples created manually (Fig. 1 shows an exam-
ple of such comparison).

In the same way as in the previous studies (Brahmi & Gall, 2006;
Gilmour & Cobus-Kuo, 2011; Homol, 2014; Kessler & Van Ullen, 2005),
different types of mistakes for individual resource types were detected.
Similarly to the Homol’s (2014) study we divided the errors found in

Table 1
Number of mistakes in bibliographical references for medical journals (Data – the number of errors in data, Punctuation – the number of errors in punctuation, Formatting – number of errors in formatting data, Average – the average number of errors
in a reference)

EndNote Mendeley RefWorks Zotero

Journal Title Data Punctuation Formatting Average Total
errors

Data Punctuation Formatting Average Total
errors

Data Punctuation Formatting Average Total
errors

Data Punctuation Formatting Average Total
errors

Annual Review of
Immunology

34 13 4 3 51 27 5 2 2 34 25 15 4 2,6 44 44 0 5 2,9 49

Anticancer Research 27 19 7 3,1 53 14 12 4 1,8 30 28 22 4 3,2 54 16 21 4 2,4 41
Blood 35 3 1 2,3 39 48 0 4 3,1 52 40 4 4 2,8 48 46 4 4 3,2 54
BMC Cancer 47 34 30 6,9 111 42 34 29 6,6 105 45 37 21 6,4 103 42 25 25 5,8 92
Investigative
ophthalmology

33 16 4 3,1 53 23 12 4 2,3 39 64 8 5 4,5 77 10 0 0 0,6 10

JAMA 26 4 3 1,9 33 12 0 0 0,7 12 23 1 0 1,4 24 7 0 0 0,4 7
Journal of Clinical
Endocrinology

26 7 16 2,9 49 21 2 18 2,4 41 45 30 33 6,4 108 14 6 18 2,2 38

Lancet 34 14 0 2,8 48 28 10 5 2,5 43 29 18 0 2,8 47 13 9 0 1,3 22
Nature Medicine 45 12 3 3,5 60 29 12 0 2,4 41 44 9 2 3,2 55 27 6 0 1,9 33
Nature Reviews
Cancer

45 12 3 3,5 60 29 12 0 2,4 41 44 9 2 3,2 55 27 6 0 1,9 33

Nature Reviews
Immunology

45 12 3 3,5 60 29 12 0 2,4 41 44 9 2 3,2 55 27 6 0 1,9 33

Nature Reviews
Neuroscience

45 12 3 3,5 60 29 12 0 2,4 41 44 9 2 3,2 55 27 6 0 1,9 33

Nature Reviews
Drug Discovery

45 12 3 3,5 60 29 12 0 2,4 41 44 9 2 3,2 55 27 6 0 1,9 33

New England
Journal of
Medicine

34 16 0 2,9 50 44 20 0 3,8 64 43 8 0 3 51 42 15 0 3,4 57

NLM style 44 4 0 2,8 48 15 0 0 0,9 15 19 15 0 2 34 6 0 0 0,4 6
Vaccine 44 3 0 2,8 47 39 0 0 2,3 39 45 27 0 4,2 72 34 0 0 2 34
TOTAL 609 193 80 52,3 882 458 155 66 40,3 679 626 230 81 55,5 937 409 110 56 34,2 575

5
9

J.K
ratoch

víl/
Th

e
Jou

rn
alofA

cadem
ic
Lib

rarian
ship

4
3
(2
017

)
57–

6
6

Table 2
Number of mistakes in bibliographical references according to the type of cited document (Data – the number of errors in data, Punctuation – the number of errors in punctuation, Formatting – number of errors in formatting data, Average – the
average number of errors in a reference)

EndNote Mendeley RefWorks Zotero

Type of a document Data Punctuation Formatting Average Total
errors

Data Punctuation Formatting Average Total
errors

Data Punctuation Formatting Average Total
errors

Data Punctuation Formatting Average Total
errors

Online article with 3 or
less authors

36 12 5 3,5 53 22 12 6 2,7 40 27 17 5 3,3 49 33 4 4 2,7 41

Online article with 7 or
more authors

62 14 6 5,5 82 25 12 6 2,9 43 30 18 5 3,5 53 36 4 5 3 45

Online book with 1
author

55 6 9 4,7 70 32 9 2 2,9 43 47 13 3 4,2 63 28 9 2 2,6 39

Online book with 2
authors

55 6 9 4,7 70 32 9 2 2,9 43 42 13 3 3,9 58 28 9 2 2,6 39

Online book with 3
authors

58 6 9 4,9 73 32 9 3 2,9 44 49 13 3 4,3 65 29 9 2 2,7 40

Online edited book 49 9 2 4 60 54 3 2 3,9 59 73 15 9 6,5 97 31 4 2 2,5 37
Web resource 80 12 2 6,3 94 63 5 3 4,7 71 66 13 2 5,4 81 50 7 3 4 60
Contribution to an
online book with 3 or
less authors

44 15 4 4,2 63 49 13 6 4,5 68 57 15 5 5,1 77 36 12 5 3,5 53

Contribution to an
online book with 3 or
more authors

44 15 4 4,2 63 50 13 5 4,5 68 56 15 5 5,1 76 37 11 5 3,5 53

Contribution to a
printed book with 3 or
less authors

12 13 4 1,9 29 22 13 6 2,7 41 31 14 5 3,3 50 27 7 5 2,6 39

Contribution to a
printed book with 3 or
more authors

12 13 4 1,9 29 23 13 5 2,7 41 30 14 5 3,3 49 28 7 4 2,6 39

Printed book with 1
author

24 9 3 2,4 36 9 9 2 1,3 20 22 7 4 2,2 33 8 5 2 1 15

Printed book with 2
authors

24 9 3 2,4 36 9 9 2 1,3 20 22 7 4 2,2 33 8 5 2 1 15

Printed book with 3
authors

24 9 3 2,4 36 9 9 2 1,3 20 22 7 4 2,2 33 8 5 2 1 15

Printed article with 3 or
less authors

0 19 5 1,6 24 0 7 6 0,9 13 1 17 5 1,5 23 3 4 4 0,7 11

Printed article with 7 or
more authors

7 17 6 2 30 2 7 6 1 15 4 18 5 1,8 27 7 4 5 1,1 16

Printed edited book 23 9 2 2,3 34 25 3 2 2 30 47 14 9 4,7 70 12 4 2 1,2 18
TOTAL 609 193 80 58,8 882 458 155 66 45,3 679 626 230 81 62,5 937 409 110 56 38,3 575

6
0

J.K
ratoch

víl/
Th

e
Jou

rn
alofA

cadem
ic
Lib

rarian
ship

4
3
(2
017

)
57–

6
6

Fig. 1. – Comparison of a manually created example bibliographic reference to an online article with 3 authors or less with references generated from EndNote and Zotero.

61J. Kratochvíl / The Journal of Academic Librarianship 43 (2017) 57–66

references into three categories. The first category comprised mistakes
in data where data was missing or redundant (e.g. the rule for the num-
ber of authors allowed in references was not followed) or the data was
incorrectly given (e.g. the wrong abbreviation for the word “editors” ap-
peared). The second category comprised mistakes in the punctuation,
such as missing or redundant punctuation, a missing or redundant
space or a symbol. The third category consists of mistakes in formatting
the data in references (wrong use of italics or bold type). After dividing
the errors into categories, the number of mistakes in each of the catego-
ries for each manager were totalled for particular journals and particular
kinds of resources. At the same time, the occurrence of mistakes in each
of the categories for the reference managers was counted.

RESULTS

In this study the bibliographical references to a web resource were
compared with those to print and online versions of a journal article, a

0
50

100
150
200
250
300
350
400

EndNote Mendeley

Fig. 2. Number of errors in bibliographic ref

book, an edited book and a contribution to it (Table 2). These references
were made using the EndNote, Mendeley, RefWorks and Zotero refer-
ence managers for 15 medical journals and the NLM style (Table 1).
For each of the journals, 17 bibliographical references were compared
with the manually-produced example. The only exception was Antican-
cer Research. In its reference instructions and randomly chosen articles,
no example of a bibliographical reference to a web resource was found.
Therefore, 16 bibliographical references were compared there. A check
of the accuracy of bibliographical references generated from the refer-
ence managers was made for 1084 bibliographical references.

From Tables 1–2 and Fig. 2, it is clear that none of the four tested ref-
erence managers were able to correctly form all bibliographical refer-
ences for any of the journals. The lowest number of mistakes (575)
was found for the Zotero reference manager. It generated the fewest
mistakes for the NLM reference style and 11 out of 15 journals (BMC
Cancer, Investigative Ophthalmology & Visual Science, JAMA, Journal
of Clinical Endocrinology, The Lancet, all Nature journals, Vaccine). The

RefWorks Zotero

erences according to the type of error.

62 J. Kratochvíl / The Journal of Academic Librarianship 43 (2017) 57–66

second lowest number of errors (679) was found in Mendeley, which
had the lowest number of mistakes compared to the other three refer-
ence managers in the references for Annual Review of Immunology
and Anticancer Research. After Zotero it showed the least number of
mistakes for a further 9 journals and the NLM style. EndNote had the
third lowest number of mistakes (882) and the fewest mistakes of all
applications for Blood and the New England Journal of Medicine, but
at the same time it showed the most mistakes for the biggest number
of journals including the NLM style. RefWorks generated the most mis-
takes in references (937), but compared to EndNote it had the most mis-
takes in references for three journals. Regarding the total number of
mistakes and the total number of journals for which the programs
made the least mistakes, the best-quality reference output came from
the program Zotero.

The Zotero program also achieved the best results in terms of the
number of bibliographical references based on the type of cited re-
sources (Table 2). Zotero generated the fewest mistakes in bibliograph-
ical references for 12 types of resources and it had the second lowest
number of mistakes for the remaining five types of resources (both ref-
erences to an online article, both references to a contribution to a print
edited book and a reference to a print article with 7 or more authors).
Here the Mendeley reference manager also produced the second lowest
number of mistakes. It had the lowest number of mistakes in references
to three types of resources and the second lowest number of mistakes in
references for 10 out of 17 types of resources. On the other hand, the
bibliographical references made by EndNote and RefWorks revealed
the most mistakes for most sources. While EndNote made most mis-
takes in references for 11 out of 17 types of sources, RefWorks had the
most errors in references for 6 types of sources. However, it is necessary
to add that EndNote was the only reference manager with the least mis-
takes for both references to a contribution to a print edited book. End-
Note also came in second place because it had the least number of
mistakes for both references to a contribution to an online edited
book. Overall, the Zotero program showed the best-quality output re-
garding the type of cited sources.

With regards to the error categories, it was found that RefWorks (99
mistakes) and EndNote (93) produced the most mistakes in references
(Table 3), while Mendeley and Zotero both produced 70 kinds of mis-
takes in references. All reference managers produced mistakes predom-
inantly in bibliographical data and RefWorks also had a markedly higher
number of mistakes in punctuation and gaps in references. In connec-
tion with the frequency of the occurrence of individual mistakes (Fig.
2), these were mostly missing data in bibliographical references, incor-
rect structure for the bibliographic data or superfluous data. To a lesser
extent there were mistakes in punctuation or incorrect formatting of
some text (e.g. bold, italic) in the reference. All four reference managers
had problems with generating the URL and the date of access in refer-
ences to online resources. EndNote did not generate the URL for 120 ref-
erences out of 135 references to online resources. For 63 references it
did not generate the date of access. For 84 references RefWorks did
not write the URL and for 68 the date of access. For 56 references
Mendeley did not write the URL and for 61 the date of access. Zotero
had the least number of such mistakes. For 34 references it did not gen-
erate the URL and the date of access. The reference managers also
showed further problems with references to online resources. For ex-
ample, EndNote generated the date of access in an incorrect form (the

Table 3
Number of types of errors in particular categories

Reference
manager

Number of
errors in data

Number of errors in
the punctuation

Number of errors in
formatting data

Total

EndNote 49 29 15 93
Mendeley 39 18 13 70
RefWorks 49 36 14 99
Zotero 42 15 13 70

month was expressed as a number rather than the month name) and
like RefWorks it did not generate information on the type of resource.
Mendeley showed a mistake in the URL (“available from” instead “at”)
and in addition it unnecessarily generated the information on the type
of resource. Zotero placed the date of access in an incorrect place in
the reference.

All four programs also had problems with generating the data on the
particular edition in references to print and online books, that is, some-
times they did not generate the information at all. EndNote did not pro-
duce the information on the edition in 51 instances out of 90 references
to print and online books. The other three managers do so for only 36 of
them. EndNote then produced an incorrect abbreviation for the word
“edition” or it gave a cardinal number rather than an ordinal number
(e.g. “2 edition” instead of “2nd ed.”) in 42 references. In other cases,
particular mistakes occurred only in some of the reference managers.
For example, Mendeley was not able to generate the year of copyright
(e.g. “c2009”) because of the letter “c” before the date. Also, some indi-
vidual reference managers generated superfluous data; EndNote added
unnecessary information on the pagination in 72 references and extra
information on the place of publication in 51 references. RefWorks
added extra information on the pagination in 39 references and super-
fluous information on the place of publishing in 36 references. Zotero
gave superfluous information on the pagination in 10 references and
gave the wrong data on editors (e.g. “Edited by” instead of “editors” in
the references for BMC Cancer) in 20 references.

In the category of mistakes concerning punctuation and whitespaces
in references, all four managers made similar mistakes in punctuation
after the name of the editor, the title of the journal and the year of pub-
lishing. The punctuation after this data was either not generated or it
was wrong (e.g. a comma instead of a full stop). In other cases there
were again individual mistakes for individual managers. For example,
EndNote, Mendeley and Zotero gave the wrong punctuation after the
names of books in some references. EndNote and RefWorks put an
extra space before the pagination in some references to journal articles,
and RefWorks did not put a bracket before the date of access to online
resources.

In the category of errors in formatting the data in bibliographical ref-
erences, all four reference managers usually showed the document titles
wrongly, whether in italics or bold type. For example, EndNote,
Mendeley and RefWorks, despite the reference instructions of BMC Can-
cer stipulating otherwise, generated the titles of contributions to an
edited book and the titles of books, journals and edited books in bold
type in references to online and print resources. Mendeley, RefWorks
and Zotero also mistakenly generated the data on authors in bold
type. This was found in references for the Journal of Clinical Endocrinol-
ogy. In addition, EndNote did not give the names of books in italics in
references to online books while it did so in print publications.

DISCUSSION

In agreement with the previous studies (Brahmi & Gall, 2006;
Homol, 2014; Kessler & Van Ullen, 2005) analysing the quality of the
bibliographical references from reference managers, the results of this
research show that none of the reference managers produces all biblio-
graphical references without mistakes. The Zotero reference manager
achieved the best results because it had remarkably excellent results
in references for the JAMA journal and the NLM style. Nonetheless,
even this reference manager as well as the other programs produced a
number of mistakes in the references to online resources. All reference
managers made on average 2.3–6.9 mistakes in producing references
to online resources. This is a high number considering the usual 9 pieces
of data in the references of an article (the author, the article title, the
journal title, the date, the volume, the number, the pagination, the
URL, the date of access), 10 pieces of data in the references of a contribu-
tion to an edited book (the author, the title of the contribution, the ed-
itor, the title of the edited book, edition, the place of publication, the

63J. Kratochvíl / The Journal of Academic Librarianship 43 (2017) 57–66

publisher, the date, the URL, the date of access) and 8 pieces of data in
the references of a book/an edited book (the author/the editor, the
title of the book/edited book, edition, the place of publication, the pub-
lisher, the date, the URL, and date of access). In this the previous findings
were confirmed. This means that the reference managers are not always
able to generate the data typical for online resources. Gilmour and
Cobus-Kuo (2011) stressed this problem in connection with the
CiteULike, Mendeley, RefWorks and Zotero reference managers. They
tested the output from these managers for the APA style. Homol
(2014) also found this for the APA style as well as the MLA style. She
compared the accuracy of bibliographical references from EBSCO Dis-
covery Service, EndNote Basic, RefWorks and Zotero. Kessler and Van
Ullen (2005) came to the same findings while testing the EasyBib, End-
Note and NoodleBib programs.

These studies put the blame for the mistakes on the reference man-
agers themselves. However, our test revealed that the mistakes are not
caused only by the program but the setting of a particular citation style.
A check of the generated references showed the following: all four refer-
ence managers generated the references to the online resources without
the URL and the date of access for the Annual Review of Immunology, but
both pieces of data were generated for the NLM style. If the same record
of the publication entered into the reference manager is generated incor-
rectly for one style and it is generated correctly for another style, then the
reference mistakes cannot be caused by the reference manager. The fault
is in setting the implemented citation styles.

The same applies to the next mistake found in this research. This was
an alleged inability of these managers to generate the titles of docu-
ments correctly. In the past the RefWorks reference manager was
blamed for this mistake (Gilmour & Cobus-Kuo, 2011; Homol, 2014)
but our research found that the reason for such mistakes is the incorrect
setting of the reference style. This means that in fact, while RefWorks
generated proper names with lowercase initial letters for the Annual
Review of Immunology and Journal of Clinical Endocrinology, the prop-
er names were generated with capital letters in the references for other
journals and the NLM style. The same error was also found in the
Mendeley and Zotero reference managers when applied to the Annual
Review of Immunology. If the reference managers generated the same
record for different journals in different ways, the program itself cannot
be at fault. There must be a mistake in the setting of the applied citation
style. Likewise, the inability to generate a shortened title of the journal
was previously taken as a mistake in the EndNote manager (Brahmi &
Gall, 2006). However, our research found that EndNote showed this
kind of problem not only with the Annual Review of Immunology and
Blood, but the abbreviated title of the journal was generated for the
other journals. The same mistake was found in the Mendeley reference
manager which was unable to generate the shortened title only for the
Journal of Clinical Endocrinology. It is again clear that the cause of the
mistakes does not necessarily have to lie in the program itself but in
the setting of the citation style. If there were a fault in the program,
the data in references for all journals rather than just some of them
would be generated incorrectly.

Some mistakes were undoubtedly also caused by the reference man-
agers themselves. Mendeley showed an inability to generate the year of
copyright in the form “c2009” because of the letter “c”. After removing
this letter, the program succeeded in generating the copyright date
without problems. At the same time the JAMA and NLM instructions,
for example, stipulate placing a letter “c” before the date in publications
containing only the year of copyright (Iverson et al., 2007; Patrias,
2007). Zotero meanwhile had a problem with generating the place of
publication and the publisher in the references to a web resource be-
cause there were no corresponding fields for this data in the record.
However, the place of publication and the publisher are data required
by the citation instructions provided in the Annual Review of Immunol-
ogy, BMC Cancer, The Lancet, the New England Journal of Medicine, Vac-
cine and the NLM style. RefWorks had problems with generating the
first name of one of the editors of the edited book. This name was

Klaus–Peter (i.e. names containing a hyphen) and RefWorks generated
only the initial letter K while the other managers wrote K–P correctly.

Aside from these two sources of mistakes in references, a third was
found as well: the directives in the reference instructions of a journal
differed from its actual practice. Our research found that, for example,
the reference instructions of Investigative Ophthalmology & Visual
Science stipulate that a title of the book be written in the normal
typeface (ARVO, 2015) while the title was actually written in italics in
the published articles. Examples include references 46 and 47 in the
introduction of the already-mentioned article on the regression of the
tumor Melanoma (Rashid et al., 2015). Vaccine does not require in its
citation instructions that the issue of the journal be given. In reality
the issue can be mentioned, which can be seen, for example, in the
article “Risk and outcomes of invasive pneumococcal disease in adults
with underlying conditions in the post–PCV 7 era, The Netherlands”
(Wagenvoort et al., 2016).

With this inconsistent approach it is understandable that some refer-
ence managers generate the references with data which do not corre-
spond with the citation instructions for the authors or with the direct
citation practice in the journal. On the other hand, mistakes were not al-
ways caused by different citation instructions and the usual citation prac-
tice. Especially in the case of print journal articles as the most often-cited
type of resources in medical journals, it was therefore surprising to find
that for more than half of journals and the NLM style, only two reference
managers were capable of generating bibliographical references to a print
article completely without any mistakes. This was the case of references
to print journal articles, generally the most cited types of documents in
medical journals, and namely references from Mendeley to Annual Re-
view of Immunology, Anticancer Research, Blood, JAMA, Nature journals,
Vaccine and the NLM style and a reference from Zotero to Blood, The
Lancet, Nature journals, the New England Journal of Medicine and the
NLM style. EndNote and RefWorks generated bibliographical references
to print articles completely without any mistakes only for JAMA and
The Lancet. EndNote also generated them for the NLM style.

Considering the above-stated facts, the question arises of which refer-
ence manager to recommend to authors intending to publish in medical
journals. It is also necessary to take into account the functionality offered
by the individual programs. Various studies (Gilmour & Cobus-Kuo, 2011;
Muldrow & Yoder, 2009; Rapchak, 2012; Robbins, 2012; Steeleworthy &
Dewan, 2013; Zhang, 2012) comparing different reference managers in-
cluding EndNote, Mendeley, RefWorks and Zotero, have been published.
The authors of these studies usually found similar functionality in all of
these applications and they usually concluded that the program’s suitabil-
ity depends on the user. Despite this, they also mentioned the positive as-
pects and downsides of the individual programs.

In the case of EndNote the ability to look up information using the
reference manager in external sources and databases and to fill in data
in online archives in EndNote Basic were evaluated positively. The
same applies to the problem-free functioning of the Cite While You
Write plugin and the accessibility of installation files with the citation
format on the websites of journals (Muldrow & Yoder, 2009; Rapchak,
2012; Zhang, 2012). The downsides, however, included the need to
have a permanent internet connection, as well as the restricted number
of external sources and databases for searching and downloading
records and the charging of a fee for the use of the program
(Steeleworthy & Dewan, 2013; Zhang, 2012). The cited studies also
mentioned the impossibility of sharing the records with other users,
which is presently possible, although the maximum number of users
is 14 (Thomson Reuters, 2014b).

Features evaluated positively in Mendeley were the ability to save
full texts and insert notes into them, the ability to also use this applica-
tion on an iPad and the free access to the program. Mendeley was
criticised for its inability to share records with other users (Gilmour &
Cobus-Kuo, 2011; Robbins, 2012; Steeleworthy & Dewan, 2013).

RefWorks was criticised the most not only because its use is charged
for but also because it requires a user to set up an account and have a

Ta
b
le

4
N
u
m
b
er

of
er
ro
rs

in
bi
bl
io
gr
ap

h
ic
al

re
fe
re
n
ce
s
to

p
ri
n
t
ar
ti
cl
es

in
m
ed

ic
al

jo
u
rn
al
s
(D

at
a
–
th
e
n
u
m
be

r
of

er
ro
rs

in
d
at
a,
Pu

n
ct
u
at
io
n
–
th
e
n
u
m
b
er

of
er
ro
rs

in
p
u
n
ct
u
at
io
n
,F
or
m
at
ti
n
g
–
n
u
m
be

r
of

er
ro
rs

in
fo
rm

at
ti
n
g
d
at
a,
A
ve

ra
ge

–
th
e
av

er
ag

e
n
u
m
be

r
of

er
ro
rs

in
a
re
fe
re
n
ce
)

E
n
d
N
o
te

M
en

d
el
ey

R
ef
W

o
rk
s

Z
o
te
ro

Jo
u
rn

al
Ti
tl
e

D
at
a

P
u
n
ct
u
at
io
n

Fo
rm

at
ti
n
g

To
ta
l
er
ro
rs

D
at
a

P
u
n
ct
u
at
io
n

Fo
rm

at
ti
n
g

To
ta
l
er
ro
rs

D
at
a

P
u
n
ct
u
at
io
n

Fo
rm

at
ti
n
g

To
ta
l
er
ro
rs

D
at
a

P
u
n
ct
u
at
io
n

Fo
rm

at
ti
n
g

To
ta
l
er
ro
rs

A
n
n
u
al

R
ev

ie
w

o
f
Im

m
u
n
o
lo
gy

2
6

0
8

0
0

0
0

0
4

0
4

1
0

1
2

A
n
ti
ca
n
ce
r
R
es
ea

rc
h

0
2

0
2

0
0

0
0

0
2

0
2

3
0

0
3

B
lo
o
d

1
0

0
1

0
0

0
0

0
0

2
2

0
0

0
0

B
M
C
C
an

ce
r

1
5

7
1
3

1
6

6
1
3

1
8

4
1
3

1
6

6
1
3

In
ve

st
ig
at
iv
e
o
p
h
th
al
m
o
lo
gy

0
6

2
8

0
4

2
6

0
4

0
4

2
0

0
2

JA
M
A

0
0

0
0

0
0

0
0

0
0

0
0

2
0

0
2

Jo
u
rn

al
o
f
C
li
n
ic
al

E
n
d
o
cr
in
o
lo
gy

0
4

2
6

0
0

2
2

2
4

4
1
0

0
2

2
4

La
n
ce
t

0
0

0
0

0
2

2
4

0
0

0
0

0
0

0
0

N
at
u
re

M
ed

ic
in
e

0
2

0
2

0
0

0
0

0
2

0
2

0
0

0
0

N
at
u
re

R
ev

ie
w
s
C
an

ce
r

0
2

0
2

0
0

0
0

0
2

0
2

0
0

0
0

N
at
u
re

R
ev

ie
w
s
Im

m
u
n
o
lo
gy

0
2

0
2

0
0

0
0

0
2

0
2

0
0

0
0

N
at
u
re

R
ev

ie
w
s
N
eu

ro
sc
ie
n
ce

0
2

0
2

0
0

0
0

0
2

0
2

0
0

0
0

N
at
u
re

R
ev

ie
w
s
D
ru
g
D
is
co

ve
ry

0
2

0
2

0
0

0
0

0
2

0
2

0
0

0
0

N
E
JM

2
2

0
4

0
2

0
2

1
2

0
3

0
0

0
0

N
LM

0
0

0
0

0
0

0
0

1
0

0
1

0
0

0
0

V
ac
ci
n
e

1
1

0
2

0
0

0
0

0
1

0
1

1
0

0
1

TO
TA

L
7

3
6

1
1

5
4

1
1
4

1
2

2
7

5
3
5

1
0

5
0

1
0

8
9

2
7

64 J. Kratochvíl / The Journal of Academic Librarianship 43 (2017) 57–66

permanent internet connection, as well as having institutional access.
Also, the number of sources and databases for searching and
downloading documents is restricted and the Write-N-Cite plugin is
cumbersome (Gilmour & Cobus-Kuo, 2011; Muldrow & Yoder, 2009;
Rapchak, 2012; Steeleworthy & Dewan, 2013). However, the possibility
to archive data online and the simple import of records from accessible
sources and databases have been evaluated positively (Muldrow &
Yoder, 2009; Steeleworthy & Dewan, 2013).

In the case of Zotero, the possibility of using the program free of charge
as well as the possibility to share records with other users, have been
evaluated positively. Moreover, Zotero has a very simple method of
downloading records from an unrestricted number of sources and data-
bases, is simple to use and allows the editing of records and the possibility
of using this reference manager on an iPad as well as Android devices
(Gilmour & Cobus-Kuo, 2011; Muldrow & Yoder, 2009; Robbins, 2012;
Steeleworthy & Dewan, 2013; Zhang, 2012). In the past, Zotero was
criticised for not being able to archive data online, the necessity of manual
copying of records for exchanging them among different devices and a
restricted number of reference styles (Muldrow & Yoder, 2009).

The above-mentioned description of the functionality of individual
reference managers is still relevant at the time of writing, except in
the case of the Zotero program. Users of Zotero presently have the pos-
sibility to obtain an online storage space after creating an account. This
online space can be synchronized with the manager installed on one’s
computer. There have also been changes in the selection of reference
styles; at the time of creating this study (February 2016), Zotero
contained 8050 citation styles that are also used by the Mendeley pro-
gram. EndNote offered 6750 styles and RefWorks 4015 citation styles
(Mendeley., 2016; RefWorks Copyright, 2009; Roy Rosenzweig Center
for History and New Media, 2016; Thomson Reuters, 2014a).

In connection with the repository, it must be noted that citation
styles are constantly evolving, frequently being updated. The citation
styles for EndNote and RefWorks are updated by the administrators of
these programs (RefWorks Copyright, 2009; Thomson Reuters, 2014a),
while the styles for Mendeley and Zotero are available in Zotero’s Style
Repository and can be updated by anyone with a knowledge of the
Citation Style Language (CSL). Administrators of Zotero’s Style Reposito-
ry avoid the risk of incorrect citation style settings by means of an
automated check of CSL source code as well as by publishing each
citation style after consulting other members of the repository on it
and approving it (Zelle, 2015a, 2015b). As the results in this study
show, Zotero’s Style Repository members devote considerable attention
to the approval process. Moreover, any registered user can display the
history of changes made to the settings of each citation style, which
makes the repository transparent.

With respect to the findings concerning the quality of bibliographical
references in this study and the technical possibilities of individual refer-
ence managers, Mendeley and Zotero seem to be the most suitable refer-
ence managers. This conclusion can be disputed, of course, as the results
of the tests include types of publications which are not often cited in med-
ical journals. Also, a comparison of the results for print articles only can
bring different results. This objection is well founded, because it is journal
articles that are mainly cited in medical journals (Barrett et al., 2016;
Delwiche, 2013; Rethlefsen & Aldrich, 2013). However, these articles are
nevertheless cited as print resources even though scientific information
sources are accessible to researchers in an online form these days. Fur-
thermore, the mentioned studies found an increase of references to
other sources published in the last 10 years compared to articles. Finally,
current medical journals as well as the NLM style and the AMA styles, as
essential citation styles for medicine, assume that reference may be
made also to other kinds of resources and they include bibliographic ref-
erence templates for these in their instructions for authors.

Despite this, additionally the analysis of the results relating to the
references of printed articles was made and showed that Mendeley
and Zotero achieve the best results with regard to the number of mis-
takes in bibliographical references to printed articles (Table 4). Although

65J. Kratochvíl / The Journal of Academic Librarianship 43 (2017) 57–66

Zotero has the least number of mistakes in total, Mendeley could be
regarded as a more accurate manager. In total Mendeley had one more
mistake in references in comparison with Zotero but it had the least num-
ber of mistakes for most journals. Mendeley created the references to ar-
ticles for the NLM style and for 10 out of 15 journals completely without
mistakes while Zotero created references completely without mistakes
for the NLM style and 8 journals. EndNote created references without
any mistakes for the NLM style and two journals. RefWorks did the
same for only two journals. It is evident then that also from the point of
view of creating bibliographical references to printed articles Mendeley
and Zotero are the reference managers which make bibliographical refer-
ences with the lowest number of mistakes.

With regard to the above-mentioned facts, Mendeley and Zotero
are the most suitable reference managers at the present time. Both of
these programs provide the same basic functionality as EndNote and
RefWorks, but they are capable of generating a larger range of biblio-
graphical references without mistakes. In addition, they are free of
charge, they have a bigger selection of reference styles and they can
be used on mobile phones. The only fundamental difference between
Mendeley and Zotero is the former’s feature for inserting notes to
downloaded full texts. Naturally, everything depends on how important
this feature is to users, and this may influence one’s decision in choosing
between Mendeley and Zotero.

CONCLUSION

Although this study has led us to recommend Mendeley or Zotero,
the conclusion is not a definitive one. Both reference managers and ref-
erence styles are under continual development and the next test might
show a different level of generated bibliographical references as well as
the functionality offered by the reference managers. In addition, the re-
sults of the study are limited by the number of journals and the refer-
ence styles used for the comparison of the references. It is not realistic
to make a complete analysis with the present number of professional
journals. This is a further reason why it is necessary to perform regular
tests to verify the ability of reference managers to generate bibliograph-
ical references with minimal mistakes. These tests should be performed
primarily by librarians who, as information professionals, can best work
with the structure of bibliographical records and data created from
them. In the past, it has repeatedly been proven (Basak, 2014;
Gilmour & Cobus-Kuo, 2011; Homol, 2014; Kessler & Van Ullen, 2005)
that a number of authors had made mistakes in references as a result
of incorrectly imported and insufficiently corrected records from differ-
ent online sources and databases. Naturally, librarians can help the au-
thors to check and correct mistakes, but a more sophisticated solution
could lie in consistent verification of reference managers, their function-
ality and the quality of the bibliographical references they generate. The
results of this study in fact reflect the need to test different reference
managers to verify their suitability for users at a particular institution.
Furthermore, while the subject of this study was to check the references
generated for medical journals, it is at the same time important to carry
out research on journals from other fields. Only in this way can librar-
ians work effectively with the present capabilities of the reference man-
agers in connection with authors’ needs. In conjunction with this, they
can adapt the range of their services, especially training on reference
managers. They can at the same time provide the necessary feedback
on the quality of the final products to the creators of reference styles.

REFERENCES

ARVO (2015). Online submission instructions for authors [Website]. (Retrieved from http://
iovs.arvojournals.org/ss/forauthors.aspx#msprep).

Bahar, Z., Beser, A., Elcigil, A., Karayurt, O., Vural, F., Ugur, O., & Kucukguclu, O. (2012). Ac-
curacy of references in eight nursing journal. HealthMed, 6(6), 2066–2073.

Barrett, A., Helwig, M., & Neves, K. (2016). Mapping the literature of hospital pharmacy.
Journal of the Medical Library Association, 104(2), 118–124. http://dx.doi.org/10.
3163/1536-5050.104.2.005.

Basak, S. K. (2014). A comparison of researcher’s reference management software:
Refworks, Mendeley, and EndNote. Journal of Economics and Behavioral Studies,
6(7), 561–568 (Retrieved from http://ifrnd.org/Research Papers/J6(7)5.pdf).

Biebuyck, J. F. (1992). Concerning the ethics and accuracy of scientific citations. The
Journal of Anesthesiology, 77(1), 1–2. http://dx.doi.org/10.1097/00000542-
199207000-00001.

Brahmi, F. A., & Gall, C. (2006). EndNote and Reference Manager citation formats com-
pared to ‘instructions to authors’ in top medical journals. Medical Reference Services
Quarterly, 25(2), 49–57. http://dx.doi.org/10.1300/J115v25n02_04.

Celayir, A. C., Sander, S., & Celayir, S. (2003). To the editor. Journal of Pediatric Surgery,
38(4), 653–654. http://dx.doi.org/10.1053/jpsu.2003.50154.

Copyright, R. W. (2009). Output style list [Website]. (Retrieved from https://www.
refworks.com/content/products/output_style.asp).

Cronin, B. (1982). Norms and functions in citation – the view of journal editors and ref-
erees in psychology. Social Science Information Studies, 2(2), 65–77. http://dx.doi.
org/10.1016/0143-6236(82)90001-1.

De Lacey, G., Record, C., & Wade, J. (1985). Style matters – how accurate are quotations
and references in medical journals. British Medical Journal, 291(6499), 884–886.
http://dx.doi.org/10.1136/bmj.291.6499.884.

Delwiche, F. A. (2013). Mapping the literature of radiation therapy. Journal of the
Medical Library Association, 101(2), 120–127. http://dx.doi.org/10.3163/1536-
5050.101.2.007.

Freimer, G. R., & Perry, M. M. (1986). Student problems with documentation. Journal of
Academic Librarianship, 11(6), 350–354.

Garfield, E. (1990). Journal editors awaken to the impact of citation errors. How we con-
trol them at ISI. Current Comments, 13(41), 367–375 (Retrieved from http://www.
garfield.library.upenn.edu/essays/v13p367y1990.pdf).

Gilmour, R., & Cobus-Kuo, L. (2011). Reference management software: A comparative
analysis of four products. Issues in Science and Technology Librarianship, 66(Summer).
http://dx.doi.org/10.5062/F4Z60KZF.

Homol, L. (2014). Web-based citation management tools: Comparing the accuracy of
their electronic journal citations. The Journal of Academic Librarianship, 40(6),
552–557. http://dx.doi.org/10.1016/j.acalib.2014.09.011.

ICMJE (2015). Recommendations for the conduct, reporting, editing, and publication of schol-
arly work in medical journals: Updated December 2015. [USA]: International Committee
of Medical Journal Editors (Retrieved from http://www.icmje.org/icmje-recommenda-
tions.pdf).

Iverson, C., Chistiansen, S., Flanagin, A., Fontanarosa, P. B., Glass, R. M., Gregoline, B., …
Young, R. K. (2007). AMA manual of style: A guide for authors and editors (10th ed ).
Oxford: Oxford University Press.

Kessler, J., & Van Ullen, M. K. (2005). Citation generators: Generating bibliographies for
the next generation. Journal of Academic Librarianship, 31(4), 310–316. http://dx.
doi.org/10.1016/j.acalib.2005.04.012.

Liu, M. (1993). Progress in documentation – the complexities of citation practice – a re-
view of citation studies. Journal of Documentation, 49(4), 370–408. http://dx.doi.org/
10.1108/eb026920.

Lok, C. K. W., Chan, M. T. V., & Martinson, I. M. (2001). Risk factors for citation errors in
peer-reviewed nursing journals. Journal of Advanced Nursing, 34(2), 223–229.
http://dx.doi.org/10.1046/j.1365-2648.2001.01748.x.

Mansfield, J. W. (1984). Why not a uniform citation system. Journal of Academic
Librarianship, 10(4), 220–222.

Mendeley. (2016). Adding new citation styles to Mendeley Desktop [Website]. (Retrieved
from http://support.mendeley.com/customer/en/portal/articles/169935-adding-
new-citation-styles-to-mendeley-desktop).

Moorthy, A. L. (1988). Towards a standard style for bibliographic references. DESIDOC
Journal of Library & Information Technology, 8(4) (Retrieved from http://publica-
tions.drdo.gov.in/ojs/index.php/djlit/article/view/2979).

Muldrow, J., & Yoder, S. (2009). Out of cite! How reference managers are taking research
to the next level. PS – Political Science and Politics, 42(1), 167–172. http://dx.doi.org/
10.1017/S1049096509090337.

Oermann, M. H., Cummings, S. L., & Wilmes, N. A. (2001). Accuracy of references in four
pediatric nursing journals. Journal of Pediatric Nursing, 16(4), 263–268. http://dx.
doi.org/10.1053/jpdn.2001.25537.

Onwuegbuzie, A., Hwang, E., Combs, J. P., & Slate, J. R. (2012). Editorial: Evidence-based
guidelines for avoiding reference list errors in manuscripts submitted to Journals
for review for publication: A replication case study of educational researcher.
Research in the Schools, 19(2), i–xvi (Retrieved from http://search.proquest.com/
docview/1509202998).

Park, S., Mardis, L. A., & Ury, C. J. (2011). I′ve lost my identity – oh, there it is… in a style
manual: Teaching citation styles and academic honesty. Reference Services Review,
39(1), 42–57. http://dx.doi.org/10.1108/00907321111108105.

Patrias, K. (2007). In D. Wendling (Ed.), Citing medicine: The NLM style guide for authors,
editors, and publishers (2nd ed ). Bethesda, MD: National Library of Medicine (Re-
trieved from http://www.ncbi.nlm.nih.gov/books/NBK7256/).

Rapchak, M. (2012). RefWorks vs Endnote: A comparison of bibliographic
management software. Nursing & Allied Health Resources Section: Newsletter,
32(1), 2–4 (Retrieved from http://nahrs.mlanet.org/home/images/newsletter/
v32no12012.pdf).

Rashid, M., Heikkonen, J., & Kivelä, T. (2015). Tumor regression after brachytherapy for
choroidal melanoma: Reduction of thickness and cross-sectional area by shape and
regression pattern. Investigative Ophthalmology & Visual Science, 56(4), 2612–2623.
http://dx.doi.org/10.1167/iovs.14-16322.

Reddy, M. S., Srinivas, S., Sabanayagam, N., & Balasubramanian, S. P. (2008). Accuracy of
references in general surgical journals – an old problem revisited. The Surgeon:
Journal of the Royal Colleges of Surgeons of Edinburgh and Ireland, 6(2), 71–75.
http://dx.doi.org/10.1016/S1479-666X(08)80067-4.

66 J. Kratochvíl / The Journal of Academic Librarianship 43 (2017) 57–66

Rethlefsen, M. L., & Aldrich, A. M. (2013). Environmental health citation patterns: Map-
ping the literature 2008–2010. Journal of the Medical Library Association, 101(1),
47–54. http://dx.doi.org/10.3163/1536-5050.101.1.008.

Robbins, L. P. (2012). Mendeley or Zotero: Which should the mobile researcher use? The
Charleston Advisor, 14(2), 5–11. http://dx.doi.org/10.5260/chara.14.2.5.

Roy Rosenzweig Center for History and New Media (2016). Zotero style repository
[Website] (Retrieved from https://www.zotero.org/styles).

Spivey, C. A., & Wilks, S. E. (2004). Reference list accuracy in social work journals.
Research on Social Work Practice, 14(4), 281–286. http://dx.doi.org/10.1177/
1049731503262131.

Steeleworthy, M., & Dewan, P. T. (2013). Web-based citation management systems:
Which one is best? Partnership: The Canadian Journal of Library and Information
Practice and Research, 8(1) (Retrieved from https://journal.lib.uoguelph.ca/index.
php/perj/article/view/2220/2780).

Sweetland, J. H. (1989). Errors in bibliographic citations: A continuing problem. The
Library Quarterly, 59(4), 291–304.

Terbille, C. (1990). Students and rules of style for reference citations. Libri, 40(3),
242–254.

Thomson Reuters (2014a). EndNote output styles [Website]. (Retrieved from http://end-
note.com/downloads/styles).

Thomson Reuters (2014b). Library sharing [Website]. (Retrieved from http://endnote.
com/product-details/library-sharing).

Wagenvoort, G. H. J., Knol, M. J., de Melker, H. E., Vlaminckx, B. J., van der Ende, A.,
Rozenbaum, M. H., & Sanders, E. A. M. (2016). Risk and outcomes of invasive pneu-
mococcal disease in adults with underlying conditions in the post-PCV7 era, The
Netherlands. Vaccine, 34(3), 334–340. http://dx.doi.org/10.1016/j.vaccine.2015.11.
048.

Wyles, D. F. (2004). Citation errors in two journals of psychiatry. Behavioral & Social
Sciences Librarian, 22(2), 27–51. http://dx.doi.org/10.1300/J103v22n02_02.

Zelle, R. M. (2015a). Criteria for accepting styles. (Retrieved from https://github.com/cita-
tion-style-language/styles/wiki/Criteria-for-Accepting-Styles).

Zelle, R. M. (2015b). Quality control. (Retrieved from https://github.com/citation-style-
language/styles/wiki/Quality-Control).

Zhang, Y. (2012). Comparison of select reference management tools. Medical Reference
Services Quarterly, 31(1), 45–60. http://dx.doi.org/10.1080/02763869.2012.641841.

  • Comparison of the Accuracy of Bibliographical References Generated for Medical Citation Styles by EndNote, Mendeley, RefWor…
    • INTRODUCTION
    • METHOD
    • RESULTS
    • DISCUSSION
    • CONCLUSION
    • REFERENCES
error: Content is protected !!