-AN-THROPOLOGICAL RECORDS 1:4 ULTURE ELEMENDISTRIBUTIONS&JVLA- .- jTHE RELIABILVY OF GULTURI 'BY i AROLD E. DRIVER S~~~ ?"X0 0 : $134v ~ ~ ~ ~ 4 lr 0 - -* StMENT:pAs . a. -., . f . , stY . < . .. . , -. . . .. . *i * # . .. I - ;S wt . ., - 5o w .. . . . .2* ? UIVE1S OF LIFOE4IA 1P'jSS rE R I~ R I 9 3 L U O R $ . - : .i-'- . 1 938 - j Q ; i 4. . *- 4 ..;2 CULTURE ELEMENT DISTRIBUTIONS: VIII THE RELIABILITY OF CULTURE ELEMENT DATA BY HAROLD E. DRIVER ANTHROPOLOGICAL RECORDS Vol. 1, No. 4 ANTHROPOLOGICAL RECORDS EDITORS: R. H. LowIE, A. L. KROEBER, R. L. OLSON Volume I, NO. 4, Pp. 205-220 Tranismitted December 8, 1936 Issued April 20, 1938 Price, 25 cents UNIVERSITY OF CALIFORNIA PRESS BERKELEY, CALIFORNIA CAMBRIDGE UNIVERSITY PRESS LONDON, ENGLAND The Uniiversity of Californiia publicationis dealinig with anithro- pological subjects are now issued in two series. The series in Americani Archaeology anid Ethniology, which was established in I903, continues unchanged in format, but is restricted to papers in which the interpretative elemenit outweighs the factual or which otherwise are of general interest. The new series, known as Anithropological Records, is issued in photolithography in a larger size. It conisists of monographs which are documenitary, of record niature, or devoted to the presen- tation primarily of niew data. MANUFACTURED IN THE UNITED STATES OF AMERICA CONTENTS Ihotion. ility of tribal totalities ....... i1ity within special topics ...... Xlity of individual elements ...... Lcal recommendations ........ y and conclusions ........... mces cited ............... Iratribal correlations from all data tratribal correlations from 706 elements mdard errors of tetrachoric R ..... itributions of least reliable elements Page . . . . . . . . . . . . 205 .. . . . . . . . . . . . 2 0 7 ... . . ........ . 209 ............. . ...... 212 . . . . . . . . . . . . 213 . . . . . .. . . . 213 .... ........ 0 . 219 TABLES 0 . . * * * 1 * * * 214 214 215 216 [ iii] CULTURE ELEMENT DISTRIBUTIONS: VIII , THE RELIABILITY OF CULTURE ELEMENT DATA 1. BY HAROLD E. DRIVER INTRODIJCTION tPrevious studies in reliability of ethnograph- 1 data are almost nonexistent. Many ethnogra- a have made brief remarks here and there t the reliability of a certain section or 'tement of a report but almost none has seri- y coped with the problem. Informants have characterized by catchwords such as good, r, reliable, unreliable, and ethnographers as observers, prejudiced observers, and the e, STe problem of the reliability of informants' tements raises at least three questions: How Sistent is the same informant in response to same questions on two or more occasions; what the amount of agreement between two or more ormants; and how near do they come to the ac- 1 facts? Material for answers to the first two ries i readily obtainable in the field, but t for he third is limited to live cultures or se for which re have exceptionally good his- ical ocuments or previous ethnographies. So fir as I knov', the best study, to date, in jiability of native informants is that made by $ois and Demetracopouloul on the mythology of 6Wintu Indians of California. They recorded perous versions of the same myths in English, (the native language, through interpreters, om the same informant on t!o occasions a year irt, from informants of varying ages, and so Oth. Their conclusions were roughly these: (1) ager versions of the same myth differed from rter versions mainly in the inclusion of songs irrelevant ethnographnic detail and in repe- tion of the same elements rather than by the ition of definitely new elements. (2) "The sence of a native audience and confidence in recorder frequently stimulate the narrator to 1 longer tales." (3) Given the same careful ,ld technique, English versions have essentially same form and content as versions recorded in native language. (4) "An able interpreter s not appreciably affect the content of a tale Imay only slightly affect the form." (5) A Ose of time does not affect a story so much as personality of the informant and circumstances tendant to the telling. Versions told a year Lrt differ no more than those told a few days rt. (6) The younger generation know far less thology than their elders, but individual dif- rences within a generation are greater than the ferences of the norms of the two generations. 11932. For complete citation see References ted. (7) There were few local differences within lW'intu territory. Although these conclusions are indeed an im- portant contribution to our knowledge of factors determining reliability of report, I believe they would have been sharpened by the use of a more definite analytical technique involving the split- ting of the various versions of the same myth into elements or incidents which could then be compared numerically. Sheer length of the tale, however, is often given by some such phrase as "one-fourth longer." The data of the present study were gathered much more rapidly and in a far less controlled manner and from fewer informants per tribe than were the Wintu myths. Nevertheless, they cover a much greater range of native culture and make possible the comparison of the reliability of va- rious topics such as subsistence, games, shaman- ism, and the like. They consist mainly of re- sponses to specific questions, whereas the Wintu myths are, of course, volunteer testimony. For these reasons, plus the difference in the nature of the material itself and the lack of general quantitative expression of the results of the lVintu study, no specific comparisons of the two will be made. The data used in this study are mainly from culture-element lists obtained by Barnett2 and myself.' Barnett worked two informants from Galice Creek in Oregon, and I two each from the Yurok, Hupa, and Karok tribes of northwestern California. Drucker filled in a column in Barnett's element list from his Tolowa data obtained pre- viously by the usual field methods, to which was matched a list gathered by me from a single Tolowva informant. I also gleaned a few more items from Drucker's manuscript where they could be equated to elements in my list which had not appeared in Barnett's. Besides these, I read Kroeber's4 ac- count of the Yurok and those of Goddard5 and Curtis' on the Hupa and entered the material (+ or -) into the element list used by me for the same tribes. All the information employed in this study has been reduced to presence (+) and absence (-) form. Quite obviously this is a simnlification of the true facts. In some contexts "+ means occur- rence among a majority of the population, as in patrilocal residence. Matrilocal residence lvas 2See Bibliography. 3Ui1D:X--Northwest California, AS 1. (In press.) 4 1925. 51903. 61924. L 205] I ANTHROPOLOGICAL RECORDS practiced to some degree by all tribes in connec- tion with "half marriage," the frequency among the Yurok being 23 per cent of 413 cases.7 Never- theless it was recorded "-" in the section on postnuptial residence. In other contexts "+" means present in any frequency at all, in some instances no doubt as low! as one or two per cent. In still other sections, an element by its very nature would be restricted to some special group, such as sucking doctors. Unfortunately this fac- tor of frequency has never been thoroughly dis- cussed by any of us who have been gathering ele- ment-list material. So far no standardization has been attempted by me throughout my entire element list, but I did try to control the factor for each element individually in the field. In gen- eral, most elements present at all were recorded "+" by me. It was only, or at least mainly, where the item implied a majority, as in postnuptial residence, that the less frequent alternative was recorded "-." I do not hesitate to admit that this simplification is a short-cut device without other merit. The majority of information in mono- graphs, however, is subject to the same limita- tions, which often are forced on the ethnographer by the limited knowFledge of informants. Concerniing absences, few informants consistent- ly deny the presence of elements unknown to them. They oftqh say, "I never saw it," "I never heard of it," /or "My father never told me about it." In such a ituation, I usually recorded minus. Wherever I felt certain that the informant was mistaken I entered a query or wrote a note ex- pressing my doubt. Where either the informant or myself seemed to be in doubt, I enclosed the en- try in parentheses. Parenthetical entries have been included in the statistics to follow, because if they are inferior items, it is desirable to know how they behave. Not infrequently an inform- ant would recognize my description or illustration of an element as belonging to a neighboring tribe and deny it for his own without hesitation. Other references to field technique will be made below from time to time. While the concrete discussion to follow is limited to four tribes, the element universe ex- tends from the Tillamook in Oregon to the Kato in California. In other words, any element present in any of the twenty-three tribes8 in this area was included in at least some statistics below. Had a wider areal universe been chosen, it would have increased the number of common absences (--) among the four tribes and raised correlation co- efficients. A narrower universe would have lowered such values. In most of the statistical treatment to follow I have compared informants with informants. The culture elements are the units counted. Were there more duplicated bodies of data available, the 7Waterman and Kroeber, 1934. 8A11 tribes in Barnett's and my own distribu- tion studies. problem might have been approached wholly from the other side, by determining how many pairs of informants agree in their responses to each ele- ment, and percentaging these values to arrive at a measure of the reliability of each element. This has been done for a part of the data but is inconclusive because of the small quantity of du plicated tribal inventories. To determine the reliability of any body of source material it is necessary to use some sta- tistical measure. It is desirable to employ the same measure to be used later on to correlate the material so that the results will have bear- ing on the intertribal or intertrait correlation Thus if one expects to apply Qp to a number of tribal inventories to obtain groupings of tribes (culture "areas"), he should use this same formu to determine the amount of correlation or assoc' ation between the responses of two informants o the same tribe. If he finds, e.g., that a number of pairs of informants show intratribal correla-, tions of about .90, this means that intertribal, correlations of the same magnitude, if any, are likely to have true values of 1.00. If it can b shown that intratribal differences, which are e rors on the part of at least one of the inforn- ants, are randomly distributed throughout a lis of elements, then all obtained correlations wil automatically be lower than their true values can then be raised to their true values by a co rection device.9 I have chosen tetrachoric r (rt),'0 computed graphically from Thurstone's diagrams,1' as a measure of the correlation between the response of pairs of informants. Most of the fourfold distributions are nearl symmetrical"2 and any other measure would give highly parallel results. Under conditions of pe fect symmetry Qp = rt. However, where an appreo able amount of asymmetry is present, I believe QB or the r included in it are more satisfactor than rt. Table 3 (see at end of this study) gives the standard error of rt. When the difference betwe two correlations is 2.5 times the standard err of the difference, ort5 + art2 the chances are approximately 99 to 1 that it is real, i.e. not owing to mere sampling error. In general, I believe it legitimate to assume that any numbe of elements is a sample of some very much larg totality. However, where the number of possibi ties is definitely limited, as in fire-making methods or arrow releases, we cannot consider 5 or so possibilities as being mere samples of very large number of alternative methods. Diff ences in informants' testimony which occur in such universes are real if only five elements 'Cf. Spearman, 1904, 1907. 10Pearson, 1900. "Chesire, Saffir, Thurstone, 1933. '2I .e-, the percentage ratio of positive to negative responses is about 50:50. c I I T I I I 206 CULTURE ELEM. DISTRIB.: VIII--DRIVER: RELIABILITY OF DATA tvolved. From a broader point of view, however, ioh elements can be considered samples of primi- Lye technology, and ultimately as samples of the al number of kinds of human cultural behavior. pling concepts seem to apply to these broad egories, not to very small ones. In some of arbitrary sections into which I have divided data of tables 3 and 4, the elements consti- ite a large part of the total number of compa- ible variants in the area: e.g., houses or games. er sections, such as ceremonies and shamanism, baps represent a smaller fraction of the total ber of possible comparable details. The reality numerical differences throughout the sections follow will be subjectively evaluated from time time, but in general I shall follow the usual pling theory. 1While the source material apparently offers Licated information on five tribes, pairs of rmants from two of these, Karok and Yurok, ong to different local groups. The first prob- is to decide whether there is any justifica- Z for assuming that the two Karok and the two ok informants respectively belong to the same tures. Qertain relationships are given in the re. lthough the two Karok groups are only slightly disparate than the two Yurok, the external tions show a definite difference. The corre- on of Karok 1 to Yurok 1 is definitely lower that of Karok 2 to Yurok 1. This means that the differences between Karok 1 and Karok 2 not randomly distributed with reference to k 1, as errors would be, but that some of behave systematically. Karok 1 is more aber- culturally as well as geographically than k 2. In contrast, the external relations of two Yurok groups to Karok 2 and Hupa 1 are ly the same. The differences between the Yurok ps are randomly distributed with reference to neighboring localities and thus behave like rs. Because of these facts I have eliminated arok from the rest of the study. Informants from the Galice and Hupa tribes are certain to belong to culturally identical local groups. The settlements of each of these tribes all fell within a 10-mile length of stream. In contrast the Yurok and Karok occupied some 40 and 50 miles of river respectively. The informants from Galice creek are blood relatives, "cousins." Those for Hupa were unrelated but were from vil- lages only 3 miles apart which joined together for certain ceremonies. For the Tolowa, Drucker used a number of informants, including the man who served as my only informant. Thus Drucker's material is perhaps less localized than my own, but this difference is slight because the Tolowa were a small group compared to the Yurok or Karok. It should be remembered throughout that the three ethnographers, Drucker, Barnett, and myself, did not work under parallel field conditions. Drucker spent 70 or 80 days with a number of Tolowa informants gathering as full material as was available. Barnett began with a relatively short and generalized element list (less than 1000 elements) constructed chiefly by Kroeber for all of California, and built this up with new elements obtained from informants to about 2000. He spent a week each with his Galice informants. I began with a list of about 3000 traits, made some inquiry into all of these, added more while in the field, and spent only three or four days with each informant. Neither time nor method of inquiry were held constant by the three field in- ve stigators. RELIABILITY OF TRIBAL TOTALITIES The totals of tables 1 and 2 summarize the chief findings. Table 1 includes all information. The values for N differ from tribe to tribe be- cause we happened to have more duplicated informa- tion on the Yurok, e.g., than on the Tolowa. The 1366 Tolowa elements are not all included in the 2337 for the Yurok, which means that the correla- tions throughout this table are based on different universes for the different tribes. These univer- ses are not strictly random samples of a larger totality, hence the correlations in this table are not strictly comparable to one another. Table 2 gives the results from the 706 elements which were positively or negatively reported for all four tribes. These correlations, made from the same universe, are wholly comparable. The correlation coefficients (rt) are these: Tribes Galice Tolowa Yurok Hupa Average Table 1 .92 .87 .91 .97 .92 Table 2 .92 .90 .97 .98 .94 t il V-1 207 ANTHROPOLOGICAL RECORDS The correlations based on table 2 are higher than those of table 1, but hardly significantly so except with respect to the Yurok. I can think of no explanation whatsoever for the difference in the two Yurok values. For the phenomenon as a whole, the apparent reason is that the shorter element list (table 2) represents elements of more general knowledge. These elements are what all ethnographers (Barnett, Drucker, and myself) thought worth including in the list of queries, and at the same time what informants would readily give answers to or sometimes volunteer. Such would expectably be more reliable. In both tables the Hupa informants show a very high reliability. The second informant was hard of hearing and slorer in response than the first. I suspect I tended to equate hesitant or uncer- tain responses to those of the first informant. I had positive assurance that both belonged to culturally identical local groups and should agree throughout. Furthermore, they were the first two informants I worked with in the area, and my knowledge of the culture and ability to control its elements was less than for the other tribes. I do not believe their true knowledge is sufficiently superior to other informants' to ac- count for this difference. Barnett's Galice creek material does not re- veal this type of bias, at least in comparison with correlations of the other three tribes. The relatively low reliability of the Tolowa is doubtless attributable to the fact that there were two ethnographers, Drucker and myself. Drucker worked without an element list, using the more usual method of inquiry where more informa- tion is volunteered. He may have erred occasion- ally in entering negatives, some of which were perhaps inferred from his general knowledge. How- ever, having read his entire manuscript, I am convinced that the vast majority of differences are my errors. Furthermore, Drucker spent some 70 or 80 days with Tolowa informants, I 3 days. Further evidences of the influence of the eth- nographer on correlations are these:13 Sources Drucker's Tolowa-Galice 2 Driver's Tolowa-Galice 2 Drucker's Tolowa-Yurok 2 Driver's Tolowa-Yurok 2 ++ +- -+ -- N rt 367 97 68 352 884 363 119 74 328 884 382 83 146 273 884 411 70 114 289 884 .84 .80 .70 .81 Our two Tolowa lists are compared with the Galice on the north and the Yurok to the south. I had worked with a Yurok before visiting the Tolowa. Drucker had not. His information on the Tolowa and that of Barnett on the Galice are essentially independent of each other although they discussed 13The raw frequencies have been included in the small tables in the text in order to give a sample of their distribution and the amount of asymmetry present. some of the Galice material at the time Barn was gathering it. My Tolowa information corro .09 higher than Drucker's with the Yurok, ani lower with the Galice. It seems obvious to me I projected previous knowledge, mainly Yurok, my Tolowa data. The fact that my informant ca from he southern part of the Tolowa territo: nea est the Yurok, and that there were a numi o0admitted Yurok influences present in small frequencies, may have caused some of the fort going differences. Further evidence of the role of the ethnol pher is given by these figures on four Hupa sources: Sources Goddard-Curtis Goddard-Driver 1 Goddard-Driver 2 Curtis-Driver 1 Curtis-Driver 2 Driver 1-Driver 2 ++ +- -+ -- N 295 6 5 27 333 274 25 5 29 333 274 25 5 29 333 270 30 8 25 333 270 30 8 25 333 274 4 4 51 333 The values .85 and .90 are not significant different because the standard error of thed ference is about .04. The value .97 is justb ly significantly higher than those of .90. Tb the significant facts are represented by the higher correlations on the one hand and the I lower ones on the other. The very high agree: between my own informants has been interprets above. Concerning the relationship between G dard's and Curtis' accounts, I am convinced Curtis drew from Goddard when he wrote his re The following tabulation of positive element mentioned in their works proves this. Goddard Niot Mentioned mentioned Mentioned 295 88 Curtis Not mentioned 228 478 Totals 523 566 The total, 1089, is the number of positive e, tries obtained by me from the Hupa. Negative were disregarded because of their infrequenc the published sources. The correlation of t fourfold table -is rt = .64. Had Goddard and Curtis randomly selected from 1089 positive ments, the correlation would have been rt t Perhaps some of this agreement in selection material i s due to the kind of information teered by informants, but I doubt if such a agreement would have come about in this way. actual terms and sentence structure of Curti conform so closely to Goddard that the indep ence of the reports could be doubted on thea grounds alone. Furthermore, Curtis' chapter 208 CULTURE ELEM. DISTRIB.: VIII--DRIVER: RELIABILITY OF DATA 6Hupa is longer than any other chapter in his on tribes in the general area. At the time ote, Goddard's report wJas practically the -published literature available. Curtis does mention names of his informants. These facts ..point to one conclusion. ithin the limits of the 333 examples cited e, what superficially appeared to be four ces thus turn out to be two. The low values, .and .90, which represent the correlations be- en these two sources are of about the same itude as those betweern other more or less in- ndent sources (totals of tables 1 and 2), and not require any special explanation. hPor fear that the 333 traits above might have a somewhat selected rather than representa- tt sample of the larger universe, I made another nt of all elements reported by Goddard and my- f. These total 613: Sources 4dard-Driver 1 idrd-Driver 2 iver 1-Driver 2 ++ +- -+ -- N rt 467 56 470 53 468 9 11 8 9 79 82 127 613 613 613 .90 .93 .99 They show no significant differences from the 8 discussed above. "Further relationships between element list and sographic sources are found by comparing Oeber s (1925) account of the Yurok with the o Yurok lists obtained by me. RELIABILITY WITHIN SPECIAL TOPICS The topical arrangement of tables 1 and 2 is the same as that used in the field except that two or three sections have been combined occasion- ally to raise the total number of elements (N) to a figure large enough to minimize sampling error. Therefore the classification is not influ- enced by any desire to "prove" preconceived theo- ries. Although the four reliability coefficients within a given class sometimes show appreciable differences, the following arrangement of averages in rank order provides a summary for the four tribes as a -ihole. Correlation (4t) Averages Topic s Ceremonies Marriage Houses Games Money, tobacco, musical instruments Weaving Counting, astronomy Death Body and dress Birth, menstruation Navigation, technology, weapons Social stratification, war Shamanism Subsistence of Various Topics Table 1 Table 2 .98 .96 .96 .95 .94 .91 .91 .89 .88 .87 .86 .85 .85 .80 .99 .98 .94 .88 .89 .85 .86 Sources ++ +- -+ -- N rt leber-Driver 1 peber-Driver 2 iver 1-Driver 2 549 43 536 58 526 56 33 29 37 206 831 .95 208 831 .94 212 831 .93 ese correlations do not differ significantly E one another. Therefore my second Yurok list probably not directly influenced by my first. identally, I interviewed the Tolowa informant teen the first and second Yurok. The lapse of ew days may have contributed to the independ- e of results. If more informants had been used it would have en possible to determine the influence of sex, , native occupation (e.g., shaman), etc., on ort. Had more field controls been employed, reliability and total quantity of volunteer rsus questioned testimony, time held constant, ald have been determined. This is difficult to complish from published literature owing to the irth of negative evidence and specific field itrols. Vithin the limits of the variation among eth- raphers and informants in this study, the per- 1l equation of the ethnographer seems to be as portant a determinant of reliability as that of native informant. The blank spaces in the table 2 column are due to the fact that no correlations were computed when N was under 50. While the range of this series of correlation averages is .18, there are no definite breaks within it. The variation in reliability of these 14 topics is not greater than that of the tribal totals discussed above, except for subsistence (.80). On the whole, material culture is no more reliable than social or religious culture. The general belief that material culture is more re- liable than other sections no doubt comes from the fact that it is preserved in museums and sometimes can be photographed in the field. A novice attempting to describe living material culture is likely to accomplish more than he would with social organization. When, however, almost the entire culture is gone, and almost all information has to be salvaged from the memo- ries of native informants, social and religious concepts can be obtained with about the same re- li-ability as material ones. The high reliability of ceremonies is explained partly by the fact that they are still given by some of the tribes. They are also completely exo- teric except for a few magical formulae recited by a priest. Furthermore, the routine of dancing and singing is repeated in identical form for :f I LI I 209 ANTHROPOLOGICAL RECORDS eight or ten days. In recent years these cere- monies have been biennial for the Yurok and Hupa instead of annual, but even so the average in- formant has seen them twenty or thirty times. The high average reliability coefficient for marriage is attributable partly to the fact that parent-in-law child-in-law avoidances have been included in this section. These avoidances, to- taling 36 traits, are entirely absent for all four tribes. Thus the common absence category of each relationship is increased by 36. This raises correlations. The elimination of avoidances would lower the average correlation to about .90. Concerning houses, a number of Yurok and Hupa structures are still standing. I filled in some of the entries from direct observation. Where the two informants were of the same local group, as at Hupa, I eliminated my own direct observations from the statistics. Under such a condition of complete preservation, informants' descriptions would also be highly accurate. Concerning subsistence, the relatively low re- liability in table 1 can be explained by at least two factors. First, minor differences in geo- graphical environment and ultra-localization by the informant undoubtedly introduce a number of real differences which are not errors on the part of either of two informants. This would apply mainly to the Yurok, although I eliminated from the count a few elements which obviously showed true local differences. Furthermore, I always be- gan my inquiry with subsistence and proceeded in the order of tables 1 and 2. Many informants, anxious to please, volunteered more information in the first few hours of the interview than dur- ing the rest of it. I tended to introduce a good part of this into the element list and expanded the subsistence section to the point of diminish- ing returns. The significantly higher reliability from the smaller sample of table 2 corroborates this interpretation. It is obvious that the quan- tity of detail concerning any topic can, be in- creased to the point where an average informant will make a high percentage of error. A small quantity of information on a given topic is also unsatisfactory because it is likely to be an in- sufficient sample of the informant's total knowl- edge and of the total number of relevant facts. The desideratum lies somerhere between. However, the relatively low correlation of two Tolowa in- formants is scarcely explainable on environmental grounds because the area was small, and neither does it fit very closely the over-expansion theory because the total number of subsistence elements is only 172 compared to 322 for the Yurok. The difference must therefore also have been caused by the particular knowledge of the ethnographers and informants involved. Such interpretations could be extended to others of these topics but would become increas- ingly subjective and have little bearing on other areas or cultures. Several other divisions of the total body of data have been made. One of these is material ob jects versus the rest of the list. Here a sharp distinction than any cited above between materia objects and behavior or belief has been made. While a topic like subsistence includes many ma- terial objects, perhaps half of its elements re~ fer to some kind of behavior or belief connecte with obtaining or preparing food. It is therefo not composed entirely of material objects. The present classification attempts to include only such elements as could be photogr phed or ob- served in museum or field without any rmati whatsoever regarding manufacture or use. Exampl all weapons are included but whether they are used in war or hunting is ruled out; hair coif- fure and tattooing are included, but the washin or greasing of hair and method of tattooing are. omitted; gaming objects are included, but not the rules of play; structural features of house are accepted but not the fact that men habituall slept in the sweat house. Tribal correlations of -such elements follow. They are compared with th totals of table 1. Pairs of informants Galice Tolowa Yurok Hupa Material objects ++ +- -+ -- N 193 28 33 217 471 257 50 35 169 511 412 63 46 279 800 422 20 13 280 735 rt .92 .85 .91 .99 Total element rt .92 .87: .91 .97 There are no significant differences between these two sets of correlations. These facts, cog bined with the ranked list of topics above, are proof that material culture elements or material objects are no more reliable than other elementt A number of illustrations were shown by me tq informants in the field. This was done for both Hupa informants, both Yurok, and the one Tolowal So far as I know, Drucker and Barnett used no illustrations. The question arises: are the re- sponses obtained from pictures more reliable tha those obtained from verbal questions alone? These are some of the relevant facts: Illustrated elements Informants ++ + + N rt Yurok 217 18 6 54 295 .95 Hupa 225 10 4 57 296 .98 The Yurok correlation here is barely signifi- cantly larger than the Yurok figure (.91) for material objects and the total list given above The Hupa remains the same. The fact that illus- trated elements show a slightly higher correla- tion for the Yurok is no proof that the picture caused this difference. A proof of the efficacy of pictures demands a control group of infoma4 on whom no pictures were used. This control grO would have to be identical in knowledge, comima 4 4 I I I I iI I i 4 i J 210 CULTURE ELEM. DISTRIB.: VIII--DRIVER: RELIABILITY OF DATA angaage, etc., with the group shown tU "ens, so that whatever difference occ Ld be attributed definitely to the pic e conditions are far from satisfied I . Nevertheless, a comparison of the r Lity of the 141 illustrated elements v -rted for all four tribes with the rel he total body of data and material ol is worth while. Cformants ++ +- -+ -- N !Galic e l Toloowa Yurok ,Hpa 62 74 108 111 5 19 7 4 8 3 3 2 66 45 23 24 141 141 141 141 he illus- eurred etures. by these reli- which are liability bjects rt .95 .91 .95 .98 the fact that the Galice correlations are also ~er for pictured elements when no pictures used eliminates the significance of the k value. From these scraps of evidence there o proof that illustrations improve reliabil- Personally, I believe they do help. No one 8 how much time either Barnett or myself t in obtaining information on these 141 items. would have to be held constant before the ieacy of pictures becomes determinant. t occurred to me that more or less generic its might show a higher reliability than more 'ific traits, the theory being that one for- ,more details than general facts. The defini- of generic versus specific is, of course, a tive one, but in order to avoid selection in i of a preconceived theory I have almost me- ically followed indentations in my original a manuscript. For example: rooden chest Cylindrical Rectanguloid In two pieces, about equal size Small opening and lid Carved decoration For feathers and valuables Phe indented elements all refer to the wooden t. A generic element, then, is one which has 'or more specific subvariants; a specific ele- , one of two or more subvariants of a generic ent. aets concerning the total number of generic specific elements derived from the entire (the universes of table 1) are: Total Generic elements elements ormant s lice iowa rok Ierage ++- +_- + __ 128 147 209 176 9 21 16 6 11 12 10 2 46 33 58 65 N 194 213 293 249 rt rt .93 .92 .80 .87 .94 .91 .99 .97 .92 .92 Specific elements Informants ++ +- -+ -- N rt Galice 255 31 63 367 716 .91 Tolowa 294 48 50 281 673 .90 Yurok 537 103 62 554 1256 .92 Hupa 503 51 30 517 1101 .97 Average .93 The only statistically significant differences in these three sets of correlations are those of the Tolowa. I have no special explanation for these differences. From the facts as a w o e, neither generic nor specific elements show any significant differences in reliability from the entire list. It occurred to me further that those elements which were widely distributed in an area might be more reliable than those of limited distribu- tion. Presumably, the more widely distributed elements would occur in higher frequencies in in- dividual tribes and hence play a more prominent r8le in the culture. To make the contrast sharp, I chose as widely distributed elements only those which were present in 11 or more of the 15" tribes in the area. Elements of limited distribu- tion are defined as those known to be absent in 11 or more of the 15 tribes but present in at least one of the four duplicate tribes. These are the findings: Informants Galice Tolowa Yurok Hipa Average Total Wide distribution elements ++ +- -+ -- N W rt W 188 9 23 21 241 .87 .76 .87 .92 263 23 5 13 304 .91 .72 .84 .87 401 19 14 1 435 .92 .03 .86 .91 419 12 2 4 437 .97 .65 .91 .97 .92 .54 .87 .92 Limited distribution Informants ++ +- -+ -- N W rt Galice Tolowa Yurok Hupa Average 9 31 21 20 4 9 28 8 5 54 12 25 36 64 3 120 72 77 149 151 .88 .73 .57 .93 .78 .82 .65 .10 .94 .63 Because of the highly asymmetrical na.ture of the fourfold distributions involved, I have com- puted the percentage of agreement, W, as well as the correlation coefficient."1 Widely distributed elements show a slightly higher percentage of agreement between pairs of informants than the total list (table 1). Elements of limited distri- bution show a definitely lower percentage of 1"The areal universe is that of table 4. The Karok and Sinkyone are counted as 2 tribes each. The rest of Barnett's material was in press, and not available to me at the time this count was made. 15 W = (a+d)/N where a is the number of elements for which both informants gave positive responses, d the number for which both gave negative re- sponses, N the total number of elements. i 211 ANTHROPOLOGICAL RECORDS agreement than the total list. Correlations give a very different picture. The total list is high- est, next elements of limited distribution, and finally those of wide distribution. Because the type of selection automatically produces asymmet- rical distributions, the meaning of the correla- tions becomes problematical. Therefore I am ac- cepting the results from W as being the more mean- ingful. Thus widely distributed elements are more reliable than those of limited distribution. To summarize the findings of this section: (1) While there are definite differences in the reliability of groups of elements arranged topi- cally, the range of variation of the reliability coefficients is no greater than that of those of the four tribal totalities, and there are no definite breaks in the series. (2) Material cul- ture or material objects show the same degree of reliability as the entire element list. (3) There is no proof that illustrations improve reliabil- ity. (4) Neither generic nor specific elements show any greater or lesser reliability than the list as a whole. (5) Widely distributed elements within a given area are apparently more reliable than narrowly distributed elements. RELIABILITY OF INDIVIDUAL ELEMENTS The reliability of an individual element can be measured in terms of the number of pairs of informants who agree in their responses to it. In the present example, when all four pairs of in- formants show intra-agreement the element is com- pletely reliable. Table 4 is a compilation of the presumably least reliable elements, those for which two, three, or four pairs of informants disagreed. The element numbers are those of my field study.'6 The universe is that of table 2, N=706. A few additional elements have been in- cluded to give supplementary information here and there. It is significant that only a single element shows four differences, and only nine show three differences. The distribution of these differ- ejices among the 706 elements is the following: Element Number of pairs of informants frequency with intradifferences 0 1 2 3 4 Actual frequency Chance frequency 477 175 44 9 455 211 36 3 The chance frequencies are those which would come about if the 294 differences were randomly scattered among the 4(706) = 2824 duplicated en- tries. I am indebted to Dr. J. M. Thompson for show- ing me the following method of deriving the chance frequencies. Let a be the probability of a difference occurring between any pair of in- formants for any element. Then: '6rDriver, MS in press. a4 = the probability that all four pair, of informants will intradiffer on the same element. 4 a3 (1-a) = the probability that exactly three pairs of informants will intra- differ on the same element. 6 a2 (1-a)2 = the probability that exactly two pairs of informants will intra-, differ on the same element. 4 a(l-a)3 = the probability that exactly one pair of info rants will intra- differ on the s-me element. (1-a)4 = the probability that no pair of informants will intradiffer on the same element. Since N = 706 elements, the total number of duplicated responses is 4(706) = 2824. The total' number of differences, by actual count, is 294. Therefore: a= 2824 =.I04 a4 4 a'(1-a) 6 a2(1-a)S 4 a (l-a)3 (l-a)4 = .000, = .004, = .052, = .299, 1.06405 1.000 which, which, which, which, which, multiplied by 706 = multiplied by 706 = multiplied by 706 = 3 multiplied by 706 = 21 multiplied by 706 = 45 70 The frequencies of elements which show two, three, or four differences are so near chance, that we have practically no proof that they are poor elements. The accumulation of differences is therefore mainly due to unknown factors whose cumulative effect produces distributions similar to those of coins or dice. Such factors might be error in recording on the part of the ethnograph verbal error in response on the part of the in- formant; or true misunderstanding on the part of either party. The fact that the questions are never given in exactly the same words by the eth nographer on two occasions means that the stimuu- lus is not fully controlled. Such control will probably never be achieved because the variety of cultures and languages involved is too great and the time too short to realize this ideal. A difference in the role played by an element in a culture may make an ethnographer's query meaning less to certain informants. In my experience wit this type of interview I have done at least halt of the talking. Each element must be described t some context before an informant will get the idea, and the ethnographer is forced to decide whether or not his description to the informant is adequate when the informant continues to re- spond negatively. Under such conditions of work, random error would be expectable. On the assumption that at least some of the ten elements for which three of four intratribal differences occurred are inferior items, I shall give my impressions regarding some of the possib i i i I I I I I I 212 CULTURE ELEM. DISTRIB.: VIII--DRIVER: RELIABILITY OF DATA B of these differences. Numbers refer to nts in table 4. Informants were doubtless confused between construction of a definite brush fence and iling of a little brush around a natural 1 or runway. This element also appears to be M typical of southern Athabascans than the th and Smith river region where systematic g was far more profitable than small game lng. 7-28. No doubt a secondary adaptation of a Wet, and only one of several ways to obtain peckers. They were also shot and snared. .Individual differences within a local p are certain to have occurred. Some persons such animals, others did not. 23. Food was nowhere habitually sold. Within ily or small village it was communal. With- it was bartered for other natural products rticles of value, occasionally dentalia. The ion lacks specificity. 550. A child's toy, hence of smnall importance. lso seems to be more typical of central Cali- ia. 648. This was no doubt of uncommon occurrence resorted to only when the girl's family was esperate need of money. 1920. There seems to have been appreciable ation in the age at vwhich children were d. There may also have been a confusion be- n a mere nickname and one formally bestowed. 1961. In many instances the girl was supposed ick up a few small sticks of wood on her way from her daily bath in the stream. She went oors at no other time. This was doubtless in- reted as work by some informants, but not by ers. 2282. Red is only one of several colors used. painting was never symbolic in northwestern ifornia, and the color used perhaps subject to 1 availability or individual preference. In general, these "worst" elements appear to unimportant or of infrequent occurrence in Mthwestern California culture. In order to tve that this is the cause of their unreliabil- one would have to determine the frequencies !unimportant" or "infrequent" items throughout entire list and see if intratribal differences eulated in greater proportions in these items in the rest of the list as a whole. This ild be a difficult and subjective procedure and hall not attempt it. nknown causes (chance) seem to be more potent Werminers of the reliability of individual items known causes. PRACTICAL RECOMENDATIONS AWhile these findings answer a few questions tcerning the reliability of culture-element a, we still know relatively little about the reliability of individual items, which is the one thing we want most to know if we are going to im- prove our prefield element lists. If even 25 per cent of the tribal element inventories already collected were duplicated by a second ethnographer and second informant, we would have material enough to come to definite conclusions. -With fifty or so duplicated lists randomly distribu d over the area of our present activity, we could - cally determine the reliability of each element with some little assurance. I see no point in un- dertaking intensive studies of the DuBois-Demet- racopoulou type to solve the element list prob- lem. These are expensive, good informants would drop dead around us while we were working with a single tribe, and the results would not be di- rectly applicable to the culture-element survey. If, from a large body of duplicated data, we still found that errors tended to be randomly distributed, we would have ample justification for correcting all obtained correlations upward to their true levels. Only local (topically as well as areally local) correlations would then differ to any important degree from their true values. SUMMARY AND CONCLUSIONS Compared with data of other social sciences, the reliability of culture-element material is fairly high. Reliability coefficients, rt, of the entire body of data examined range from .87 to .97. Percentages of agreement vary from .84 to .91. The low values are likely to be nearer the true reliability because higher ones are from data collected by a single ethnographer who may have possessed a bias of some kind. Within the limits of personal variation among the present ethnographers and informants, the - personal equation of the ethnographer seems to be as important as that of the informant. Although certain topics are definitely more reliable than others, the range of variation is little greater than that of total tribal inven- tories. No one major division of culture, material, social, or religious, shows any higher reliability than any other. Material objects show the same reliability as the entire body of data. Illustrations apparently do not increase re- liability. Generic or specific elements show no signifi- cant differences in reliability from each other or from the data as a whole. Widely distributed elements are apparently more reliable than the unselected elements in the list; narrowly distributed elements less reliable. Individual elements show few demonstrable differences in reliability. There is a need for more duplicated source material if we are to learn more about the kind and cause of differences between informants and ethnographers. 213 AN'THROPOLOGICAL RECORDS TABLE 1 Intratribal Correlations from All Data (viz., Galice Inf't 1 with Galice Inf't 2, etc.) a) C., 0 - rAS 0 Correlations (rt) Number of elements (N) 0 , --I U ~ - 0 -1) s O o od N .H 0 0 aS 0 A. 4 O. , rH Eo 9 m 4 C9 EH P0 ' Subs is tence Houses Navigation, technology, weapons Body and dress Weaving Money, tobacco, musical instruments Games Counting,' astronomy Marriage Birth, menstruation Death Social stratification, war Shamanism Ceremonies Total .87 .63 .74 .95 .80 162 172 322 315' .96 .91 .98 1.00 .96 177 192 226 88 .74 .82 .89 .98 .86 126 131 240 222 .96 .67 .89 .99 .88 93 94 208 196' .77 .96 1.00 .91 57 46 105 105 .98 .92 .88 .98 .94 73 64 84 84 1.00 .96 .96 .89 .95 89 85 155 120 .84 .98 .91 19 15 76 56 1.00 .97 .89 .98 .96 91 94 142 138 .84 .80 .92 .92 .87 134 121 203 166 .82 .96 .83 .93 .89 79 85 142 123 .92 .56 .95 .96 .85 104 56 126 100 .88 .63 .96 .93 .85 132 94 171 146 .97 1.00 .96 .98 124 117 137 * .92 .87 .91 .97 .92 1460 1366 2337 1859 I witnessed the White Deerskin Dance and obtained information about other ceremonies from onl one informant. TABLE 2 Intratrib.al Correlations from 706 Elements Correlations (rt) N 0 ~ c) W oH o :~ ~~. 0 0 gj i E- Subsistence .86 .75 .88 .94 .86 93 Houses 18 Navigation, technology, weapons .76 .85 .93 1.00 .89 82 Body and dress .98 .86 .92 1.00 .94 56 Weaving 35 Money, tobacco, musical instruments 49 Games 1.00 .98 1.00 .95 .98 64 Counting, astronomy 6 Marriage 1.00 1.00 .97 1.00 .99 69 Birth, menstruation .84 .86 .97 .85 .88 71 Death 40 Social stratification, war 31 Shamanism 1.00 .44 .99 .97 .85 50 Ceremonies 42 Total .92 .90 .97 .98 .94 706 Immonama I I 214 CULTURE ELEM. DISTRIB.: VIII--DRIVER: RELIABILITY OF DATA TABLE 3 Standard Errors of Tetrachoric R (Standard errors of rt, ort, when asymmetry*= .00 for both variables. These increase at an in- ging rate as asymmetry of either variable increases. They are increased 50 per cent when asym- * ? .70 for both variables, and are doubled when asymetry = ? .82 for both variables. Compiled Pearson, 1913.) Correlation Coefficient, rt .40 .45 .50 .55 .60 .65 .70 .75 .80 .85 .90 .95 .99 .197 .190 .181 .172 .162 .151 .137 .124 .107 .089 .068 .042 .015 .160 .154 .148 .140 .132 .123 .112 .101 .088 .073 .055 .034 .011 .139 .134 .128 .122 .115 .107 .097 .087 .076 063 .048 .030 .009 .113 .109 .105 .099 .093 .067 .079 .071 .062 .051 .039 .024 .008 .098 .095 .091 .086 .081 .075 .069 .062 .054 .045 .034 .021 .007 .081 .078 .074 .071 .066 .062 .057 .051 .044 .037 .028 .017 .005 .070 .067 .065 .061 .058 .054 .049 .044 .038 .032 .024 .015 .005 .052 .060 .057 .054 .051 .048 .043 .039 .034 .028 .021 .013 .004 .052 .050 .048 .046 .043 .040 .037 .033 .029 .024 .018 .011 .003 .044 .043 .041 .039 .037 .034 .031 .028 .024 .020 .015 .009 .003 .036 .035 .033 .032 .030 .028 .025 .023 .020 .016 .013 .008 .002 .031 .030 .029 .027 .025 .024 .022 .019 .017 .014 .011 .007 .002 * Isymmetry is definied as - 1.00 where plus refers to the positive responses. : r 215 ANTHROPOLOGICAL RECORDS TABLE 4 Distributions of Least Reliable Elements (+, present; -, absent; Gal 1, 2, Barnett's first and second Galice informants; Tol Di, Driver'i Tolowa; Tol Du, Drucker's Tolowa; Yur 1, 2, Driver's first and second Yurok informants; Hup 1, 2, Driver's first and second Hupa informants; numerals to left of columns of + and - entries = number of pairs informants who differed.) Subsistence Hunting 1. Driving into fence with nooses in gaps. S. Rabbits .................... 6. QGail ..................... 8. Driving with fire 11. Small game ................... 13. Driving into water .................. 27. Nets, bag type .................... 28. Woodpeckers .................. 39. Deadfalls ...................... 44. For large game ................. 45. For small game ................. Animals Eaten 88. Ursus horribilis eaten ................ 89. Felis cougar eaten .................. Fishing 175. Gill net ....................... 190. Crab-claw rattle on net ............... Various 423. Food sold .......... Houses 588. Notched plank or log ladder . 589. Type b house ...... Technology 747. V.'ood meat platter ........ 3 2 2 2 3 3 2 2 3 2 2 2 4 2 2 WNeapons 879. Wooden arrow straightener . 880. Perforated. 881. Forked stick ..... 895. Ogiver carried at side under 896. QDuiver carried on back . . . 907-8. Slings used ........ 906. For hunting, by men 909. For war ....... 910. As boy's toy only . . 922. Elkhide helmet ...... * e * e * @ arm * @ * @ * * * @ * e * e 2 2 2 2 2 - ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ - S E~~~~~~~~~~~~~~~~~~~~~~~~~~~4~~~~~~~~~~~~~~~~~~~~~~~~~~~ r- C\2 (::: A r C\l r- C%2 / Ib It Iq Ivg g + + + + + + + + + + + + + + + + + + + + + + ? + + - + + + + + - + + + + + + + + + + + + + + ; + + + ? + + + + + + - + - + - + - + + - + + + - + + + + + + + + + + + + + + + + + + + + I I I~~. I___ I 216 + + , + +i + , + + + +: + + +I +:+ + 1 + + +I +. I -1 CULTURE ELEM. DISTRIB.: VIII--DRIVER: RELIABILITY OF DATA TABLE 4 -- Continued rHi rH H r i i g 03 0 0 _: I _________________________________________ 0 0 E-4 E-44~~~~~~~~~~~~~~~~~~~~~~~~~~E- E- 4A ; O-l = 11. 86. 04. 87. Body and Dress Soaproot brush for hair Cap of fur for men .... Hide shirt ........ 995. Buckskin .... Breechclout (between legs) 1002. Buckskin . .... Leggings. 1038. Buckskin ..... Weaving *63. Deep sifting of winnowing basket, pointed bottom 41. Net mesh spacer of wood ............... 42. Net mesh spacer of bone or horn ........... Musical Instruments ,11. Hide drum .............. -1. Bull-roarer ............. 1345-6. Toy ............. Game s Hand, Grass, Many Stick Game 2. Two bones or sticks per player ............ 1454. Of hollow bone ................ 64. Hide in grass in hand ................ 1545-6. Jacks ............ 47. With stones . . . . . . . . . . . . . . . . . . . . Tops 50. Acorn ........................ Marriage 47. Child betrothal before puberty .... 1648. With payment by groom s side Birth 23. Birth aided by drinking vegetable concoction ..... Milk Teeth 11. Thrown away ay.................... 1916. Over house .................. Name -O. Given soon after birth (up to six months) 3 I 217 2 2 2 2 2 2 2 2 + + + + + + + + + + + ? ? + ? + + + + + + + + + + .X+ + + + + + 2 2 2 2 2 + + + - + - + - + - + - + - + - + - + - - + + + + _ - + - + _ _ - + + - + + - + + - + + + + 2 2 2 2 2 3 3 + + - + + - + - + - + - + - + - + - + + - + - + + + - + + + + + + - + 2 2 L fl, vI i? IV w .1I t I L I I I PI I . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 0 .PO. Given soon after birth (up to six months) . 3 + + + ANTHROPOLOGICAL RECORDS TABLE 4 -- Concluded L 1- I H C i i Girl's Puberty 1948. Covered or veiled when going outside 1952. Looking at people taboo ...... 1961. Work compulsory, getting wood . . . 1999. Men and women dance separately . . . Death 5 2 3 2 2083. Canoe of dead broken ................. 2084. House of dead burned ................. War 2281-2. War paint ...................... 2282. Red. 2283. Black ..................... 2284. Wlhite. 2293. Prisoners enslaved .................. 2326. Dance of incitement: abreast ............. Shamanism 2424. Power from human spirit or ghost ........... 2431. Power from reptiles .. ............ 2 2 3 2 2 2 2 2 218 + + + - + _ + - + + _ + + + + + + + + + + + + + + + + + 1 + 4" -4 + , +4 +4 + I + 1 + + + + + + + + + + - + + 4 -- I I I I~~~~~~~~~~~~~~ CULTURE ELEM. DISTRIB.: VIII--DRIVER: RELIABILITY OF DATA REFERENCES CITED Bibliographical abbreviations used American Anthropologist. Bureau of American Ethnology, Bulletins. Journal of the Royal Anthropological Institute \Great Pritain). University of California, Anthropological Records. University of California, Publications in American Archaeology and Ethnology. tt, H. G. 7. Culture Element Distributions:VII-- Oregon Coast. UC-AR 1:155-204. re, L., Saffir, M., Thurstone, L. L. p33. Computing Diagrams for the Tetrachoric Correlation Coefficient. University of Chicago Bookstore. is, E. S. 924. The North American' Indian. Vol. 13. or, H. E. a press. Culture Element Distributions: X-- Northwest California Area. UC-AR. ker, Philip 937. The Tolowa and their Southwest Oregon Kin. UC-PAAE 36:221-300. is, Cora, and Demetracopoulou, Dorothy 932. A Study of Wintu Mythology. Journal of American Folk-lore, 45:375-500. ird, Pliny Earle 303. Life and Culture of the Hupa. UC-PAAE 1:1-88. Kroeber, A. L. 1925. Handbook of the Indians of C lifornia. BAE-B 78. Pearson, Karl 1900. On the Correlation of Characters Not Quantitatively Measurable. Philosophical Transactions 195A:1-47. The original derivation of rt. 1913. On the Probable Error of a Coefficient of Correlation as Found from a Four-fold Table. Biometrika 9:22-27. Spearman, C. 1904. The Proof and Measurement of the Associ- ation between Two Things. Amer. Jour. Psychol. 15:72-101. 1907. Demonstration of Formulae for True Measure of Correlation. Amer. Jour. Psychol. 18:161-69. Waterman, T. T., and Kroeber, A. L. 1934. Yurok Marriages. UC-PAAE 35:1-14. AA BAE-B JRMI UC-AR UC- PAAE 219