【正文】 = (.2)(.5)+(.8)(.5) = .5 Pobs Pchance Kappa = 1 Pchance =(.)/() = .2 .8 .5 .5 Bad Good Appraiser 1 Good Bad Appraiser 2 Pobs = .5 + .2 = .7 3/10 = .3 2/10 = .2 0/10 = 0 5/10 = .5 Add Add Add Add How is this interpreted? 7 Measurement Systems Analysis (MSA) Attribute How To Interpret Kappa Results The general rule for interpreting Kappa results are as follows: 解釋Kappa的規(guī)則通常如下 ? 0 – Nonrandom disagreement 非隨機(jī)的不一致 ? – Measurement System needs attention 測(cè)量系統(tǒng)需關(guān)注 ? – Generally acceptable, improvement may be needed depending on application and risk 通常可接受，根據(jù)需要和 風(fēng)險(xiǎn)進(jìn)行改善 ? – Excellent Measurement System 極好的測(cè)量系統(tǒng) Remember, consider your Measurement System and how well the above criteria might apply. 8 Measurement Systems Analysis (MSA) Attribute Attribute Data – Kappa Exercise 1. Your pany produces documents that are filled with alphanumeric characters. If a document has one or more numerals (09), it is defective. Your mission is to identify the defective documents./你公司的產(chǎn)品是字母與數(shù)字混合編排的文件。如果文件中有 1個(gè)或多個(gè)數(shù)字 (09)，就是不合格品。你的任務(wù)就是識(shí)別不合格的文件。 2. In teams of two, appoint one person to be the data collector – The other person is the inspector. The data collection sheet is located on the following page. 2人一組，任命其中一個(gè)為數(shù)據(jù)收集員，另一個(gè)是檢查員，數(shù)據(jù)收集表在后面一頁(yè)中 3. The document number is located under each of the 20 documents. The documents will be visible for three seconds before automatically advancing. The exercise will be finished in 60 seconds (three seconds/part).共有 20個(gè)文件，每個(gè)顯示 3秒，然后自動(dòng)進(jìn)行下一個(gè)。練習(xí)在 60s內(nèi)完成 (3s/部件 ). 4. Each Appraiser will perform two trials 每個(gè)評(píng)價(jià)者重復(fù) 2次 5. Perform analysis in MINITAB174。 per the next slides 進(jìn)行 MTB分析 9 Measurement Systems Analysis (MSA) Attribute S V Q O 6 Q N I T Q X N Z I Z B M Q T P Z H J K M 23 Sample Kappa Exercise Begin Kappa exercise 10 Measurement Systems Analysis (MSA) Attribute Kappa Exercise – Data Collection Form P a r t N o .T r i a l 1 ( P / F )T r i a l 2 ( P / F )A g r e e / D i s a g r e e ?T r i a l 1 ( P / F )T r i a l 2 ( P / F )A g r e e / D i s a g r e e ?J u d g e 1 T r i a l 1J u d g e 2 T r i a l 2A g r e e / D i s a g r e e ?1234567891011121314151617181920C o m p a r e J u d g e sJ u d g e 1 J u d g e 2NOTE: Part count of 20 used only for demonstration. Sample size guidelines found on page 13. 11 Measurement Systems Analysis (MSA) Attribute Kappa Analysis In MINITAB Put data into MINITAB, each judges trial in a separate column To analyze, go to Stat ? Quality Tools ? Attribute Agreement Analysis 1 2 3 Enter Judges (Appraisers) and quantity Select “Results” button and click last option 4 12 Measurement Systems Analysis (MSA) Attribute Attribute Gage RR Study Attribute Gage RR Study for J1T1, J1T2, J2T1, J2T2 Within Appraiser Assessment Agreement Appraiser Inspected Matched Percent (%) % CI 1 20 19 ( , ) 2 20 18 ( , ) Matched: Appraiser agrees with him/herself across trials. Kappa Statistics Appraiser Response Kappa SE Kappa Z P(vs. 0) 1 b g 2 b g Between Appraisers Assessment Agreement Inspected Matched Percent (%) % CI 20 18 ( , ) Matched: All Appraisers39。 assessments agree with each other. Kappa Statistics Response Kappa SE Kappa Z P(vs. 0) b g MINITAB Output – Session Window Since Kappa for both Appraisers, both agree well with themselves/ 2個(gè)評(píng)價(jià)者 Kappa ，一致度較好 ? Adequate repeatability/足夠的重復(fù)性 Since Kappa for the Between Appraisers,/ 評(píng)價(jià)者間的 Kappa ? Adequate reproducibility/足夠的再現(xiàn)性 13 Measurement Systems Analysis (MSA) Attribute Guidelines For Kappa Studies ? Planning 策劃 Sample Size樣本大小 ? 100 samples with two trials per Appraiser / 100個(gè)樣品，每個(gè)評(píng)價(jià)者重復(fù) 2次 ? If only 50100 samples are available, do three trials per Appraiser/如果只有 50100個(gè)樣品，每個(gè)評(píng)價(jià)者重復(fù) 3次 ? 50, understand that you may need very high Kappas to have adequate confidence/50, 需要很高的Kappa值才能有足夠的信賴(lài)性 Sample Part Selection 樣品選擇 ? Parts in the study should represent the full range of variation – Practically speaking, if you chose “really good” parts and “really bad” parts, you wouldn’t be testing the Measurement Systems ability to accurately categorize the ones in between/研究的樣品能夠代表變差的整個(gè)范圍 –事實(shí)上，如果選擇 ”很好 ”或”很壞”的部件，就不能檢驗(yàn)測(cè)量系統(tǒng)正確分類(lèi)的能力 ? For maximum confidence in the calculated Kappa, we would like to have 50/50 mix of good/bad parts。 30/70 ratio is acceptable – Beyond this level, single dis