【正文】 nt Systems Analysis (MSA) Attribute Guidelines For Kappa Studies ? Planning 策劃 Sample Size樣本大小 ? 100 samples with two trials per Appraiser / 100個樣品，每個評價者重復(fù) 2次 ? If only 50100 samples are available, do three trials per Appraiser/如果只有 50100個樣品，每個評價者重復(fù) 3次 ? 50, understand that you may need very high Kappas to have adequate confidence/50, 需要很高的Kappa值才能有足夠的信賴性 Sample Part Selection 樣品選擇 ? Parts in the study should represent the full range of variation – Practically speaking, if you chose “really good” parts and “really bad” parts, you wouldn’t be testing the Measurement Systems ability to accurately categorize the ones in between/研究的樣品能夠代表變差的整個范圍 –事實上，如果選擇 ”很好 ”或”很壞”的部件，就不能檢驗測量系統(tǒng)正確分類的能力 ? For maximum confidence in the calculated Kappa, we would like to have 50/50 mix of good/bad parts。 or use a numeric scale such as 15 /順序型數(shù)據(jù)是指 – 分類是 3個或 3個以上，例如嚴(yán)重不一致、不一致、不確定、一致、非常一致，或者用數(shù)字表示如 15 ? When the attribute data can be represented by three or more categories that can be arranged in a rank order, Kendall’s Coefficient of Concordance (KCC) can be used to evaluate the Measurement System /屬性數(shù)據(jù)用 3個或以上分類的順序表示，可用 KCC評價測量系統(tǒng) ? KCC ranges from 0 to 1 / KCC 范圍從 01 ? Unlike Kappa, KCC does not treat misclassifications equally, ., the difference between a “mild” and “medium” is less than the difference between “mild” and “very hot” / KCC不同于Kappa，它不能處理誤分類，如： “mild” 和“ medium” 的差異小于 “mild” and “very hot” 的差異 KAPPA: Pass/Fail KCC: Mild/Medium/Hot/Very Hot (Hot sauce) Attribute MSA With Ordered Categories – Kendall’s Coefficient Of Concordance 0 1 Strong association No association Let’s walk through an example to see how these are calculated. 16 Measurement Systems Analysis (MSA) Attribute Kendall’s Coefficient Of Concordance Example Three judges score the quality of a proposal. The scale they use is 15, 1 being “poor”, 5 being “excellent.” The results from their scoring are provided in the following table. / 下面是一個提議好壞的判斷值。 assessments agree with each other. Fleiss39。s Coefficient of Concordance Coef Chi Sq DF P 5 MINITAB Output – Session Window You still get the Kappa Statistics, but you won’t need to use them here/還是能得到Kappa統(tǒng)計表，但是用不到它們 First we need to look at the pvalue. Remember, KCC 0, indicates association, if the pvalue is small, we accept that KCC 0. The pvalue tells you the probability that some association was found purely by chance. This is saying that there is a % chance a nonzero KCC (some level of association) was found by chance alone. You decide based on the amount of risk you’re willing to take. A value like is typical. / 首先要看 p值，記住 KCC0，表示有聯(lián)系，如果p值很小，我們接受 KCC0。這里表示有的純粹由于偶然而發(fā)生非 0 KCC的機(jī)會是 %(互相聯(lián)系的水平 )。通常設(shè)為 The KCC is fairly low here, see criteria on next slide… 這個 KCC比較小，看下一頁的標(biāo)準(zhǔn) 20 Measurement Systems Analysis (MSA) Attribute How To Interpret Kendall’s Coefficient Of Concordance ? In general, KCC can vary from to ? The higher the value, the higher degree association among the assessments made by the Appraisers /值越大，評價者的一致性越高 ? How close to is needed?需要和 ？ Although very situational dependent, the general guidelines are as follows:雖然有很多不同狀況，但是一般規(guī)則如下 – Low degree of association, Measurement System needs attention 低程度聯(lián)系，需關(guān)注測量系統(tǒng) – Generally acceptable 通?？山邮? As stated on the prior page, the pvalue for KCC should also be low, generally less than – This reduces your risk of getting an acceptable KCC just by random ， KCC的 p值應(yīng)該較小，通常比 – 減少由于偶然造成接收的 KCC的風(fēng)險 21 Measurement Systems Analysis (MSA) Attribute Guidelines For Kendall’s Studies ? Planning 策劃 Sample size 樣本大小 ? More is better – As your sample size increases, your confidence intervals around your KCC decrease 越多越好 – 樣本大小增大，和 KCC有關(guān)的置信區(qū)間范圍減小 ? Collect as many samples as practically possible, 20 minimum is a guideline, ≥ 30 is best / 收集盡可能多的樣品，至少 20，最好 ≥ 30 ? Perform at least two trials per Appraiser 每個評價者至少重復(fù) 2次 Sample part selection 樣品選擇 ? Parts in the study should represent the full range of variation and thus utilize the full range of the rating scale 研究的樣品能代表變差的整個范圍，這樣就能使用整個范圍的分類標(biāo)準(zhǔn) ? Execution 實施 Parts should be rated in random order independently (no parisons) /部件應(yīng)該以隨機(jī)順序評價 Study should be blind研究應(yīng)該是盲測 Rating time should be similar to that “normally” used評價時間也和 正常使用近似 22 Measurement Systems Analysis (MSA) Attribute Guidelines For Kendall’s Studies (Cont’d) ? Analysis 分析 Prior to reviewing the KCC value, check to see that the pvalue is low (generally ) – If it is not, add more samples to the study or add another trial / 在看 KCC值之前，檢查 p值是否較小 (通常 ) – 如不是，增加樣品或增加 1次重復(fù) Review the repeatability portion first (Within Appraiser), if an Appraiser’s KCC is very low, they may need improvement (see Improvement below) / 先檢查重復(fù)性部分 (評價者內(nèi) ),如果評價者的 KCC很低，那就需要改進(jìn)(看下面的改進(jìn) ) For Appraisers that have acceptable