Week 3 – Discussion: Educational Measurement
Respond to the following prompts in the Educational Measurement discussion forum by Wednesday 11/01/2023
Select two types of evidenced-based educational assessment, how can these resources add to your core foundational knowledge for implementing practices and strategies in research and program evaluation? ?
In what situation(s) would observations and interviews be used to collect data in educational settings and/or educational practices? How would the data support effective practices at the individual, group, and/or systems level?
Review the Foundation for Educational Measurement Download Foundation for Educational Measurement presentation. —-> See Attachment for reading
Informative Links:
https://content.acsa.org/category/technology/
Foundation for Educational Measurement
PPS 6025: Methodology of Educational Research
Created by Helen Y. Sung, Ph.D.
Types of Sampling for Quantitative Studies
Purpose:
Sampling for quantitative studies is to obtain a group of participants who will represent a larger group.
The degree of representativeness and the quality of information is influenced by the sampling techniques used.
Population:
Quantitative study studies need to generalize to a well-defined larger group. This larger group is called a population.
Target population refers to the specification and can be described conceptually or in broad terms.
Probability Sampling:
Subjects are selected randomly so that the researching knows the probability of selecting each member of the population.
It implies that each member of the population as a whole has an equal chance of being selected.
Margin of error:
It is an interval in which true population value lie.
There is always a margin of error in sampling and it must be considered in interpreting the results.
Example in political polls, 45%+/_ 3%.
Example:
Target population – all middle school teachers in California
Identify the sampling frame- teachers from State Department of Education.
Identify the sample size –select 10% of teachers
Select method of sampling-stratified random, sampling by grade level.
Select sample- select specific teacher for sample
Sampling
Simple random:
This method is often used with small number of cases.
Systematic Sampling – Every person has a number or a name that can be drawn from a hat.
Systematic:
Every nth element is selected from the list of all elements in the population.
Stratified Sampling:
First divide into homogeneous subgroups, then select from the subgroup using simple random or systematic procedures.
Cluster Sampling:
Where there is no exhaustive list of all individuals, cluster sampling is used. Random selection of naturally occurring groups or clusters.
Most common way of random sample of a large population is computer selected.
Sampling Continued
Nonprobability Sampling:
Not always feasible, necessary or desirable to obtain a probability sample.
Not every element in the population has a chance of being selected.
In journals, a group of subjects has not been selected for larger population
Qualitative studies use nonprobability sampling exclusively.
Convenience Sampling:
Subjects are selected because of availability.
The purpose is to not generalize but to understand the relationships that may exist.
It is reasonable to interpret the result as valid for those similar to those studied.
Quota Sampling:
It is used when the researcher is unable to take a probability sample but still wants a sample that is representative of the entire population.
Subjects are selected nonrandomly according to specific criteria.
geographic area or types of community
Measurement
Definition:
The assignment of numbers to indicate different values of a variables.
Measurement is used to determine how much of a trait, attribute or characteristic and individual possesses.
Purpose of measurement:
To obtain information about the variables that are being studied.
In education, it includes variables such as intelligence, achievement, aptitude, classroom environment, attitudes, and values.
Evaluation is interchangeably used with measurement. It is used to make decisions.
Assessment:
Specific process of diagnosing of individual difficulties.
Frequency Distribution
Widely used to interpret intelligence and achievement scores. It is used to explain assessment results to parents, teachers, administrators. Diagnostic criteria are based on the significance of scatter.
Symmetrical and bell shaped
A bell shaped curve is use to indicate a normal curve.
The interpretation of conative assessment as usually based on a normative curve.
Majority of the scores tend to cluster around the middle with same number of scores above and below the middle point.
Because the normal curve characterize many naturally occurring phenomena, it is used extensively for research and statistical procedures
Measures of Central Tendency
Mode- frequency
Median-middle score of the distribution with 50% above and 50% below
Mean-arithmetic average, add all the scores and divide the sum by the number of scores.
Measure of Variability
Show how spread out the distribution of score is from the mean, or how much scatter exists in the distribution. It can be in arranged from highest to lowest.
Standard Deviation:
The measure of variability most often used in research.
It indicates the average distance score from the mean of the distribution. Hence, it is also referred to as the average deviation.
In a normal distribution 68% falls between one standard deviation ± from the mean. Refer to page 141 in your book under the ‘normative probability curve.’
In cognitive assessments, mean=100, SD=15.
Percentile Rank:
Indicates the percentage of scores at or below a particular score.
Be prepared to convert standard scores to percentile rank.
Correlation:
Measures a relationship between two variables.
Positive correlation-increase in one variable lead to increase in another variable
Negative correlation- increase in one variable, the other variable decrease
Correlation coefficient:
Linear relationship between the two variables, r =+/-1
Validity of Measurement
The degree to which an instrument measure what is says it measures. It is the judgment of the appropriateness of a measure for the specific inference. Inference is valid or invalid, not the measure. (Example, cognitive assessment on African American)
Test Content:
Sample of items or questions in the instrument is representative of some appropriate universe or domain of content or tasks.
So experts in the area examine the contents to see if it meets the objectives or criteria.
Content validity test contains items that measures certain traits.
Internal Structure:
Refers to how items are related to each other and how different parts of an instrument are related.
Relationship between items and parts of the instruments are empirically consistent with the theory.
Different components of intelligence test
Measure of Variability continued
Relations to other variables:
Showing how scores from a given measure relate to similar as well as different traits.
Construct validity:
Test captures the characteristics of the trait.
Response processes:
In order to enhance reliability standard conditions of data collection need to be established.
Consequence of testing:
Observed behaviors during testing.
Response process and consequence of testing is related to behaviors observed during testing such as attentive, follows direction, test anxiety, establish rapport, motivated to do well, exert good effort, slow to respond, etc.
Types of Sampling Procedures
Purposeful:
Researcher selects particular individuals or cases because they will be informative about the topic.
Researcher’s knowledge about the population determines the judgment about inclusion of cases that will be information rich.
Typical Case:
The researcher investigates a person, group or site that is ‘typical’ or representative of many.
Extreme Case:
Often used to compare outlier, unique or atypical cases compared to others.
Type of Sampling Procedures continued
Maximum Variation:
Greatest heterogeneity among individuals, groups, or cases to represent both ends of the continuum of values on the characteristic of interest.
Snowball:
Researcher begins with few participants and then ask them to recommend others who are known to have desired profile, attributes or characteristics.
Critical case:
Phenomenon of interest is illustrated by individuals, groups, or sites in unique ways.
It uses these situations as an opportunity to learn and understand.
The Affect of Sampling
Volunteers:
Tend to be better educated, higher socioeconomic, more intelligent, more sociable, more unconventional, less conforming, less authoritarian than non-volunteers.
The nature of results depends on the types of person who respond.
Sample size:
Of at least 30 subjects for correlational research, at least 15 subjects in each group for experimental research.
In purpose of sampling is to gain in-depth information, sampling is considered complete when no new information is divulged, called saturation.
Subject motivation:
Subjects might be motivated to respond in certain way.
Subjects are likely to respond favorably because they had used the program.
The Affect of Sampling continued
Sampling bias:
Researcher deliberately selects subjects who will respond in a particular way to support a point or result.
Response variability:
Important goal of sampling in quantitative study is to obtain subjects who represent adequate variability on the measure that are used.
Sufficient variability is needed so that differences and relationships can be detected.
chapter test vs. state test
image1.png
The post Select two types of evidenced-based educational assessment, how can these resources add to your core foundational knowledge for implementing practices and strategies in research and p first appeared on Writeden.