科学素养测评 (1).ppt

《科学素养测评 (1).ppt》由会员分享，可在线阅读，更多相关《科学素养测评 (1).ppt（60页珍藏版）》请在淘文阁 - 分享文档赚钱的网站上搜索。

1、The Rasch ModelDavid FortusThe Rasch ModelA model developed by Georg Rasch that uses ordinal data collected by items to estimate individual levels of traits,and at the same time,the difficulty of the items used to assess those traits.Test Results11ItemsRaw ScorePersonsabcdefghijklA1110000110016B1011

2、000010004C1110100111119D1011000010015E0110010111118F1111000110017G1010010010116H1010000000013I1111000110017J11111111101010K1010010010116L0110010111118M0000000000000N11111111111112Difficulty361811812621173Bond,T.G.,&Fox,C.M.(2001).Applying the Rasch model:Fundamental measurement in the human sciences

3、.Mahwah,NJ:Lawrence Erlbaum.Test Results Rearranged According to Personal Ability11ItemsRaw ScorePersonsabcdefghijklN11111111111112J11111111101010C1110100111119E0110010111118L0110010111118F1111000110017I1111000110017A1110000110016G1010010010116K1010010010116D1011000010015B1011000010004H1010000000013

4、M0000000000000Difficulty361811812621173Test Results Rearranged According to Personal Ability and Item DifficultyItemsAbilityPersonscialbhkdfjegN11111111111112J11101111101110C1111111001109E1101111011008L1101111011008I1111110100007F1111110100007K1111001010006A1111110000006G1111001010006D1111000100005B

5、1110000100004H1011000000003M0000000000000Difficulty123366788111112Participants Whose Knowledge Cannot be Measured by the TestItemsAbilityPersonscialbhkdfjegN11111111111112J11101111101110C1111111001109E1101111011008L1101111011008I1111110100007F1111110100007K1111001010006A1111110000006G1111001010006D1

6、111000100005B1110000100004H1011000000003M0000000000000Difficulty123366788111112Participants Whose Knowledge Cannot be Measured by the TestItemsAbilityPersonscialbhkdfjegN11111111111112J11101111101110C1111111001109E1101111011008L1101111011008I1111110100007F1111110100007K1111001010006A1111110000006G11

7、11001010006D1111000100005B1110000100004H1011000000003M0000000000000Difficulty123366788111112Only Participants Whose Knowledge Lies Within the Range of the TestItemsAbilityPersonscialbhkdfjegJ11101111101110C1111111001109E1101111011008L1101111011008I1111110100007F1111110100007K1111001010006A1111110000

8、006G1111001010006D1111000100005B1110000100004H1011000000003Difficulty01225567791011Items that are Too EasyItemsAbilityPersonscialbhkdfjegJ11101111101110C1111111001109E1101111011008L1101111011008I1111110100007F1111110100007K1111001010006A1111110000006G1111001010006D1111000100005B1110000100004H1011000

9、000003Difficulty01225567791011Matched Items and ParticipantsItemsAbilityPersonsialbhkdfjegJ110111110119C111111001108E101111011007L101111011007I111110100006F111110100006K111001010005A111110000005G111001010005D111000100004B110000100003H011000000002Difficulty1225567791011Normalized Abilities and Diffic

10、ultiesItemsAbility(m/12)%PersonsialbhkdfjegJ1101111101182C1111110011073E1011110110064L1011110110064I1111101000055F1111101000055K1110010100045A1111100000045G1110010100045D1110001000036B1100001000027H0110000000018Difficulty(n/11)%817174242505858758392Unexpected ResultsItemsAbility m/12%Personsialbhkdf

11、jegJ1101111101182C1111110011073E1011110110064L1011110110064I1111101000055F1111101000055K1110010100045A1111100000045G1110010100045D1110001000036B1100001000027H0110000000018Difficulty(n/11%)817174242505858758392Thurstone(1925):Results of Binet ItemsThurstone(1925):Characteristic CurveProbabilityStuden

12、t Ability&ProbabilityAssume a student has an ability.There is a probability that a student of a given ability will be able to correctly answer a given test item P().0 P()1051015-5-10-1510.90.80.70.60.50.40.30.20.10P()Item Difficulty Shifting Curve Left&Right051015-5-10-1510.90.80.70.60.50.40.30.20.1

13、0Item Difficulty Shifting Curve Left&Right051015-5-10-1510.90.80.70.60.50.40.30.20.10Discriminatory Power:Changing Steepness of Curve051015-5-10-1510.90.80.70.60.50.40.30.20.10Logistic Function is the student abilityai is discriminatory power of the itemi is the difficulty of the itemThe Rasch Model

14、Assumes all items have identical discriminatory power ai =1When does P(,)=?10.50P()=Rotating the Graph10.50-10-8-6-4-20246810P()P()for Items of Different DifficultyIncreasing DifficultyDecreasing Difficulty1Each Curve can be Defined by a Single Point(The curves dont intersect)Increasing DifficultyDe

15、creasing Difficulty21=12=2Item Difficulties1234Increasing DifficultyDecreasing DifficultyWright Maps1234Increasing DifficultyDecreasing DifficultyxxxxxxxxxxxxxxxxxxxxxIncreasing AbilityDecreasing AbilityxxxiWright Map of Test DataWright Map of Test DataWright Map of Test DataThe distance between a r

16、espondent and an item(-)determines the probability that the respondent will answer the item correctly.If the respondent is above the item,this probability is greater than 50%.If the respondent is below the item,this probability is less than 50%.Does this Test Provide Adequate Coverage for the Range

17、of Abilities?Remember that an item discriminates the best between respondents whose ability lies near the items difficulty.051015-5-10-1510.90.80.70.60.50.40.30.20.10Does this Test Provide Adequate Coverage for the Range of Abilities?Remember that an item discriminates the best between respondents w

18、hose ability lies near the items difficulty.How are the Abilities and Difficulties Calculated?1.A program assumes all the items have identical difficulty i=0;it uses LMS to calculate the abilities that best fit the data.2.The program fixes the abilities and uses LMS to best fit the difficulties.It t

19、hen fixes the difficulties and recalculates the abilities,and so on until the change between one step and the next is negligible.How to Use with Open-Ended Items?This girl sees the tree.Draw arrows to show how thelight from the sun helps her to see the tree.(From IQWST)ResponsePointsaOne arrow from

20、sun to tree,second arrow from tree to girl2bArrow from sun to tree only1cArrow from tree to girl only1dIf only arrow from sun to girl is present0eIf any arrow from girl to tree or girl to sun is present 0fArrow from tree to sun,0yCrossed out/erased,or impossible to interpret,lines but no arrows0zBla

21、nk 0Thresholds12 threshold 2nd threshold01 1st threshold0All IQWST ItemsLikert-type Items Generate Ordinal DataI am so afraid of computers I avoid using themTD D N A TA1 2 3 4 5If Likert-type Items Generated Interval DataI am so afraid of computers I avoid using themTD D N A TA 1 2 3 4 5Averaging Li

22、kert-type ItemsI am so afraid of computers I avoid using themTD D N A TAIm afraid I will make mistakes when using my computerTD D N A TA Less Anxious More AnxiousAvoid using TD D N A TAMistakesTD D N A TA Thresholds1TAthreshold4th threshold0A3rd thresholdN2nd thresholdD1st thresholdTDWright Map for

23、a Questionnaire Based on Likert-type ItemsA Different Kind of Wright MapInfit&OutfitFit statistics indicate how accurately the model fits the dataMost widely used are outfit and infitBoth are based on residuals=the differences between observations(ni)and expected values(Ei)In the dichotomous model,o

24、bservations are either 0 or 1.Expected values are 0 P(,)1The residual is always 0Infit&OutfitInfit&OutfitUsing Rasch Analysis to Develop an Instrument:An ExampleThis example is taken from:Fortus,D.,&Vedder-Weiss,D.(2014).Measuring students continuing motivation for science learning.Journal of Resear

25、ch in Science Teaching,51(4),497-522.Please read the article to get the full details of the process.Placing Items on a ContinuumWe developed 19 Likert-type items based upon the literature that dealt with students after-school activities.How did we develop them?For that you need to read the articleBa

26、sed on motivational theory and our understanding of children,we hypothesized the location of the items on a continuum,from those that measured rejection of science to those that measured engagement with science.We did this to test the alignment between theory and the empirical results we would recei

27、ve.Our OrderingRejection of extra-curricular science activitiesEmbracement of extra-curricular science activities1521319101894145,71116171,36,812Gathering DataWe gathered data from about 1000 studentsThe data was used to build a Rasch model.We received the following Wright mapWright MapGetting Rid o

28、f Superfluous and Poorly Functioning ItemsAll items that had poor infit or outfit were removed.A new Rasch model for the data was built using only the remaining itemsThe Wright map indicated that the items covered the entire breadth of student abilities.However,there were several items whose Thursto

29、nian thresholds overlapped with others.We looked for items that,if removed,would not leave gaps in the metric.We removed all except for 7 items and reanalyzed the dataNew Wright MapNew OrderingComparing the New Order with the Original OrderIt appeared that we had originally placed all the final 7 it

30、ems in their correct order,except for one item 13.We zoomed in on this item to try and understand why it did not behave as predicted by theory.This led to suggested changes to motivational theory.Summary of Items CharacteristicsItem#DifficultyInfitThurstonian Thresholds122334451.661.05-.38.43.991.629.211.12-1.08-0.18.581.52101.051.07-.09.791.362.1611Rev-.731.10-1.84-1.20-.53.6013.461.21-.34.24.681.2715Rev-1.201.15-2.15-1.56-1.03-.0817Rev-.46.89-1.32-.72-.27.48Thank you for your attention.I hope you enjoyed the lecture and feel that you learned something useful from it!