Reliability

If a test is unreliable, then although the results for one use may actually be valid, for another they may be invalid. Reliability is thus a measure of how much you can trust the results of a test. Tests often have high reliability but at the expense of validity. In other words, you can get the same result, time after time, but it does not tell you what you really want to know. Reliability is a measure of the consistency with which a predictor continues to predict performance with the same degree of success. That means that, for instance, two interviews at a different time and place, with different interviewers and questions but under otherwise same conditions and with the same applicants will bring the same result; namely the best candidate should still be the best and the interviewees who failed should still fail. It is also possible to maintain the conditions, the applicants and the structure but to change the other parameters of the assessment. By comparing the results, information about the reliability can be gained. However it is difficult to conduct these tests due to several constraints. It is considered to be nearly impossible to guarantee equal conditions for each sequence, as well as to provide sets of questions with different formulations but in a similar context. Furthermore the applicants have to be willing to take part a second time. These are only a few factors out of a bundle of problems which arise by testing reliability. Reliability is a prerequisite for validity, this means that it is necessary, but not sufficient to ensure validity. This is easily understandable because: if a test is so unreliable that it produces two different estimates of a persons present behaviour, how can we believe that it gives a good estimate of future behaviour? In the selection context, validity refers to the extent to which performance on the selection device/test is associated with performance on the job. Thus, the degree to which the technique produces consistent results is Reliability where as consistency is a measure of reliability through similarity within the test, with individual questions giving predictable answers every time. For example, if an applicant was being interviewed by two managers for a job in two separate interviews, the interview technique should provide some data so that the interviewers agreed with each other about the applicant as an individual. Alternatively, if a number of candidates are given the same selection test, the test should provide consistent results concerning individual differences between candidates. The statistical analysis of selection techniques normally provides a reliability coefficient which if closer to 1.0, more dependable the technique. Reliability improves as we increase the number of relevant items that are combined to generate a value. If we wanted to measure maths ability, we could give all candidates one math problem. We would then be able to separate our pool into two groups; those that answered correctly and those that did not. If we were to ask two problems, we would be able to sort the group into four groups and we would have more confidence that the group that answered both questions

There are several types to estimate reliability. 1. Test-Retest Reliability: is the correlation between two successive measurements with the same test. For example, you can give your test in the morning to your pilot sample and then again in the afternoon. The two sets of data should be highly correlated if the test is reliable. The pilot sample should theoretically answer the same way if nothing has changed. 2. Equivalent Forms Reliability: is the successive administration of two parallel forms of the same test. A good example is the SAT. There are two versions that measure Verbal and Math skills. Two forms for measuring Math should be highly correlated and that would document reliability. 3. Split Half Reliability: is when, for example, you have the SAT Math test and divide the items on it in two parts. If you correlated the first half of the items with the second half of the items, they should be highly correlated if they are reliable. 4. Internal Consistency Reliability: is when only one form of the test is available, or you can ensure that the items are homogeneous or all measuring the same construct. 5.Inter-Rater or Inter-Observer Reliability: Used to assess the degree to which different raters/observers give consistent estimates of the same phenomenon.