Reflection on Presentation 3: Instrumentation Part 1 (Questionnaires)

On August 12, 2010, we entered the world of “Instrumentation” presented by Zubaidah Abdul Ghani, Norela Elias and Roslina Ahmed Tajuddin which covered largely on Questionnaire. The presentation was straightforward and informative. The examples given by the presenters managed to shed some light to my understanding on Instrumentation.  

The definition of data, instrumentation, validity, reliability, objectivity and usability is summarized in the table below:

 

As discussed in the class, there are three (3) methods for obtaining information, which are (1) Researcher instrument (2) Subject instrument and (3) Informants instruments. Researcher instrument (RI) is obtained by directly or indirectly assessing the subjects of a study. For example, in a Statistics class, a researcher observes students, examine students’ records, and noting the frequency of oral assessments. Subjects instrument (SI) is obtained through a self-report data that are provided by the subject of the study themselves. For instance, a researcher might have to request students’ products (essays) for evidence or interview the students.  Another method of obtaining data is Informants instrument (II) which is data provided by other people about the subjects of a study. For example, a researcher interviews teachers or students and assess each student’s thinking skills based on their prior experience.

The instruments for Researcher-Completed Instruments and Subject-Completed Instruments are as in the table below:

 

Generally, most of quantitative data obtained are reported in the form of scores. There are two scores namely Raw Scores and Derived Scores.

Raw Scores – Refers to the initial score obtained. It can be the total number of items an individual gets correct or answers in a certain way on a test. However, assessing an individual raw score is difficult to interpret and has little meaning. Examples of raw scores would be a student’s marks: Bahasa Malaysia (90), English (53), Science (89) and Mathematics (91).

Derived Scores – Refers to the data obtained by taking raw scores and then convert them into more useful scores on some type of standardized basis. It is able to indicate a particular individual’s raw score in relation to other raw scores in the same distribution and able to help the researchers compare scores between individuals taking the same test. The three examples of derived scores are (1) Age & Grade-Level Equivalent Scores (2) Percentile Ranks Scores and (3) Standard Scores.

As what has been discussed in the classroom, when we want to use the scores, we must ensure that the reference group make sense. For instance, it is quite misleading if we want to compare girls’ and boys’ performance on Mathematics subject. The group that used to determine derived scores is called the norm group and instruments that provide such scores are referred to as norm-referred instrument. Another example is: Semester 2 students of Educational Leadership & Management (Norm Group) and Statistics Test (Norm-Referenced Instruments).

Another alternative to performance or achievement instrument is Criterion-Referenced Instruments. It focuses more on instruction rather than evaluating learner’s progress through gain scores. Yet, it is based on specific goals or target for each learner to achieve. So, why use this test? The main reason would be to determine whether or not the candidate has the demonstrated mastery of a certain skill or set of skills. For example, consider a student of Education program going for a practical training at school for 3 months. The result would be either pass or fail the assessment.

In the class, I recall that Measurement Scale is the foundation of scientific investigation. Measurement is the assignments of numbers to objects. The 4 types of measurements are Nominal (groups and label the data only and reports frequencies or percentages), Ordinal (ranks data, use number only to indicate ranking), Interval (assumes that equal differences between scores mean equal differences in the variable measured) and Ratio (a ratio scale that has a true zero point and does not have negative value).

Lastly, as discussed in the presentation, the data must be prepared for analysis consistently in order to generate accurate conclusion. The data is normally coded and tabulated in Computer Software such as SPSS and ANOVA.

Till then, thanks for reading! To get the Power Point slides, click "Instrumentation Part I- Questionnaire".