Evaluation in Knowledge Organiza

1. Traditional approaches to knowledge organization.	Systems are evaluated on the basis of how well they reflect literary warrant as related to current scientific standards. Subject experts used to construe/revise/evaluate systems. (Some resemblance with peer-review).
1a. Business- or management like approaches	Systems are evaluated on the basis of management criteria: What are the costs and benefits from a management perspective. This perspective also emphasis standardization in KO
2. The facet-analytic approach	Systems are evaluated on the basis of their logical principles and how consistent those principles have been applied.
3. Information retrieval tradition	Systems are evaluated on the basis of experiments, e.g. the measuring of recall and precision.
4. User oriented views and Cognitive view in KO	Systems are evaluated on the basis of user preferences or on the basis of cognitive models.
5. Bibliometric approaches	(The same as Information retrieval tradition?)
6. The domain Analytic approach	Systems are evaluated on the basis of literary warrant, how they reflect domain specific knowledge emphasizing how they relate to different views and epistemologies in the knowledge represented. Also pragmatic analyses concerning what goals, interests and consequences are involved.

Wikipedia has been compared with Encyclopedia Britannica in an article in Nature. See Giles (2005), Encyclopedia Britannica (2006) and Nature's response March 23, 2006. The method used for comparing those two knowledge organization systems is the peer-review system known form scientific journals. The reviewers were asked to look for three types of inaccuracy: factual errors, critical omissions and misleading statements. Such criteria are somewhat different from criteria, which should be used for evaluating semantic tools such as classification systems and thesauri. In those cases are the most important dimensions 1) the coverage (are all important concepts and terms included or are there systematic biases? 2) How are the terms defined? 3) Which semantic relations are provided?

Supplementary information: http://www.nature.com/nature/journal/v438/n7070/extref/438900a-s1.doc