WEBVTT 1 00:00:01.710 --> 00:00:05.520 Hello and welcome to the video lecture on experiments 2 00:00:05.520 --> 00:00:08.793 and the use of experiments in social science research. 3 00:00:09.900 --> 00:00:14.067 So we're going to do an introduction to tell you about, 4 00:00:18.840 --> 00:00:21.210 for the purpose of this class, 5 00:00:21.210 --> 00:00:26.210 what are the three elements of a classical experiment. 6 00:00:26.550 --> 00:00:31.320 We're gonna talk a bit about how to select subjects, 7 00:00:31.320 --> 00:00:34.170 some of the validity issues in experiments, 8 00:00:34.170 --> 00:00:36.970 and last, what are their strengths and their weaknesses? 9 00:00:39.300 --> 00:00:41.430 So there's many, many ways 10 00:00:41.430 --> 00:00:45.750 in which we informally do experiment. 11 00:00:45.750 --> 00:00:49.200 We do something and we see what the actions are. 12 00:00:49.200 --> 00:00:53.010 We taste the soup and see if it needs more spice. 13 00:00:53.010 --> 00:00:54.030 We go outside 14 00:00:54.030 --> 00:00:59.030 and see if we need an extra layer or hat or gloves. 15 00:01:02.520 --> 00:01:05.400 But in social science research, 16 00:01:05.400 --> 00:01:10.050 they tend to be very sort of narrow in scope, 17 00:01:10.050 --> 00:01:13.350 that we really want to narrow down on a cause 18 00:01:13.350 --> 00:01:14.523 and an effect. 19 00:01:15.390 --> 00:01:18.180 It's not good for exploratory... 20 00:01:18.180 --> 00:01:21.150 It's good for sort of seeing the effect 21 00:01:21.150 --> 00:01:25.260 of a specific intervention, 22 00:01:25.260 --> 00:01:27.900 like people who are exposed to some things 23 00:01:27.900 --> 00:01:32.073 are more or less likely to take some behavior. 24 00:01:33.990 --> 00:01:36.000 So the three parts, 25 00:01:36.000 --> 00:01:37.470 and we'll go through each one, 26 00:01:37.470 --> 00:01:40.380 are independent and dependent variables, 27 00:01:40.380 --> 00:01:41.610 pre and post testing, 28 00:01:41.610 --> 00:01:43.953 and having experimental and control groups. 29 00:01:44.880 --> 00:01:47.970 So one way of thinking about it 30 00:01:47.970 --> 00:01:52.500 is the independent is the cause. 31 00:01:52.500 --> 00:01:57.360 What is the treatment or the stimulus? 32 00:01:57.360 --> 00:02:00.780 And in some cases it either may happen 33 00:02:00.780 --> 00:02:02.190 or it doesn't happen, 34 00:02:02.190 --> 00:02:06.840 and the dependent variable is the response. 35 00:02:06.840 --> 00:02:11.037 What's the sort of behavior that we are looking at 36 00:02:14.249 --> 00:02:15.933 as a result or as a response. 37 00:02:18.780 --> 00:02:23.780 The second element is pre and post-tests, 38 00:02:24.180 --> 00:02:26.230 where you measure 39 00:02:27.150 --> 00:02:31.600 sort of where someone is via a pretest, 40 00:02:34.140 --> 00:02:37.530 you expose them to some sort of treatment 41 00:02:37.530 --> 00:02:41.310 and then you post-test, 42 00:02:41.310 --> 00:02:42.963 and see if there's a change. 43 00:02:46.680 --> 00:02:50.190 Last is the experimental and the control group, 44 00:02:50.190 --> 00:02:53.010 where the experimental group gets it 45 00:02:53.010 --> 00:02:56.730 and the control doesn't. 46 00:02:56.730 --> 00:03:00.360 Think about, why should the experimental group 47 00:03:00.360 --> 00:03:02.670 resemble the control groups? 48 00:03:02.670 --> 00:03:06.900 Why should these groups sort of resemble each other 49 00:03:06.900 --> 00:03:08.850 in all feasible ways? 50 00:03:08.850 --> 00:03:11.670 And why even have a control group? 51 00:03:11.670 --> 00:03:15.510 Why not just do a test, 52 00:03:15.510 --> 00:03:17.250 do an intervention, 53 00:03:17.250 --> 00:03:19.800 do another test, see if there's a change? 54 00:03:19.800 --> 00:03:21.873 Why do you need an intervention? 55 00:03:24.900 --> 00:03:27.330 What would be an example 56 00:03:27.330 --> 00:03:31.780 of how we could use this in our class project? 57 00:03:32.640 --> 00:03:35.610 What would be each of these things? 58 00:03:35.610 --> 00:03:38.163 And we'll discuss this in class. 59 00:03:40.590 --> 00:03:44.253 In many cases you do a double-blind, 60 00:03:46.320 --> 00:03:50.070 where the researcher doesn't know 61 00:03:50.070 --> 00:03:54.090 what's the control group and what is the experimental group, 62 00:03:54.090 --> 00:03:57.060 and they often do this in medicine, 63 00:03:57.060 --> 00:04:00.210 where one group gets a pill, 64 00:04:00.210 --> 00:04:03.483 say that they think will actually treat the condition, 65 00:04:05.600 --> 00:04:09.113 and the other control group gets a placebo, 66 00:04:09.990 --> 00:04:14.853 and neither group knows who is who and why does that matter? 67 00:04:16.110 --> 00:04:18.270 What would be the effect? 68 00:04:18.270 --> 00:04:21.930 You know, it's clear that if I know I'm getting a pill, 69 00:04:21.930 --> 00:04:24.930 that I'll think I'll get better and I get better, 70 00:04:24.930 --> 00:04:26.400 and if I don't get it, I'll get worse. 71 00:04:26.400 --> 00:04:29.700 But why is it important that even the doctors 72 00:04:29.700 --> 00:04:33.120 or the nurses not know? 73 00:04:33.120 --> 00:04:35.010 When would it be important? 74 00:04:35.010 --> 00:04:39.330 What are some of the behaviors that the doctor 75 00:04:39.330 --> 00:04:42.933 or the nurse say, might take, that would mess things up? 76 00:04:48.600 --> 00:04:51.120 Very often, folks like you, 77 00:04:51.120 --> 00:04:55.260 college students, are used in selecting subjects. 78 00:04:55.260 --> 00:04:58.200 What are the strengths and weaknesses of this? 79 00:04:58.200 --> 00:05:01.800 The strengths are that you're available, 80 00:05:01.800 --> 00:05:06.060 you're probably curious 81 00:05:06.060 --> 00:05:08.430 and it might sound like fun, 82 00:05:08.430 --> 00:05:10.953 and also if we pay you, 83 00:05:12.000 --> 00:05:15.330 many of you would appreciate the money. 84 00:05:15.330 --> 00:05:17.730 The weakness is, 85 00:05:17.730 --> 00:05:22.543 there's no way to generalize to larger populations. 86 00:05:28.680 --> 00:05:32.490 Experiments can have validity issues. 87 00:05:32.490 --> 00:05:33.480 Can you really... 88 00:05:33.480 --> 00:05:36.690 So like internally, 89 00:05:36.690 --> 00:05:41.350 can we really sort of draw the conclusion 90 00:05:43.763 --> 00:05:45.693 that this treatment had an effect? 91 00:05:46.770 --> 00:05:51.570 Sometimes there can be historical events 92 00:05:51.570 --> 00:05:56.010 which sort of blow everything out of the water. 93 00:05:56.010 --> 00:06:01.010 So if had done pre and post tests on contagious diseases 94 00:06:02.190 --> 00:06:05.650 and right in the middle of the treatment 95 00:06:09.630 --> 00:06:13.230 COVID or another pandemic hit, you know, 96 00:06:13.230 --> 00:06:16.200 that sort of blows out of the water 97 00:06:16.200 --> 00:06:19.470 Any sort of like training course. 98 00:06:19.470 --> 00:06:22.710 Same with flood preparedness. 99 00:06:22.710 --> 00:06:26.070 Here in Vermont we've had many, many floods, 100 00:06:26.070 --> 00:06:28.200 like so-called (indistinct) floods. 101 00:06:28.200 --> 00:06:32.250 So the effect of watching a video say, 102 00:06:32.250 --> 00:06:37.250 on flood preparedness would not compare to so many farmers 103 00:06:37.470 --> 00:06:40.353 and others who actually lost their home. 104 00:06:42.758 --> 00:06:46.860 And there's just sort of the natural maturation 105 00:06:46.860 --> 00:06:48.630 of your subjects, 106 00:06:48.630 --> 00:06:53.370 as well as sometimes, the test itself might change, 107 00:06:53.370 --> 00:06:55.680 that we might learn that this just wasn't 108 00:06:55.680 --> 00:06:57.240 a good way to do it, 109 00:06:57.240 --> 00:07:00.543 and we sort of change the course in midstream. 110 00:07:04.950 --> 00:07:08.430 I have used experimental auctions 111 00:07:08.430 --> 00:07:12.360 to measure how much people will pay 112 00:07:12.360 --> 00:07:16.020 for certain product attributes. 113 00:07:16.020 --> 00:07:21.020 And there's sort of the artificiality of it, 114 00:07:21.690 --> 00:07:24.540 that there's like a wealth effect. 115 00:07:24.540 --> 00:07:27.390 Do you spend sort of the funny money 116 00:07:27.390 --> 00:07:29.430 that you get in an experiment 117 00:07:29.430 --> 00:07:34.290 in the same way that you would say, in a grocery store? 118 00:07:34.290 --> 00:07:38.460 And as with an auction, 119 00:07:38.460 --> 00:07:41.820 like, we liked to win and we liked to get the thing 120 00:07:41.820 --> 00:07:45.600 and will this auction really reveal 121 00:07:45.600 --> 00:07:47.970 what somebody will pay? 122 00:07:47.970 --> 00:07:49.950 And not, "I just kept bidding 123 00:07:49.950 --> 00:07:54.357 so that I would win, because it's fun to win." 124 00:07:57.420 --> 00:08:00.150 There also can be a concern 125 00:08:00.150 --> 00:08:05.150 that the pretest itself can be a type of treatment 126 00:08:06.720 --> 00:08:08.313 or stimulus. 127 00:08:12.540 --> 00:08:17.540 And here is that the so-called Solomon Four Group design, 128 00:08:17.940 --> 00:08:22.940 where only half of the group gets a pretest. 129 00:08:28.411 --> 00:08:30.830 So there's sort of two treatments, 130 00:08:33.690 --> 00:08:37.590 the actual sort of pretest itself, 131 00:08:37.590 --> 00:08:41.847 the actual treatment itself, but also the pretest itself. 132 00:08:47.250 --> 00:08:52.250 So for example, if I gave a pretest 133 00:08:55.770 --> 00:08:58.423 on statistical knowledge 134 00:09:04.650 --> 00:09:08.130 and then went and gave one group, 135 00:09:08.130 --> 00:09:11.850 half of the class, some sort of tutorial, 136 00:09:11.850 --> 00:09:14.160 and one group didn't get it, 137 00:09:14.160 --> 00:09:19.160 that if you did very poorly on the pretest, 138 00:09:24.090 --> 00:09:27.660 you might sort of, you know, work a lot harder, 139 00:09:27.660 --> 00:09:31.740 study more, get outside resources, 140 00:09:31.740 --> 00:09:34.140 as opposed to those who did well, 141 00:09:34.140 --> 00:09:39.140 and any effect of the change from pre to post-test 142 00:09:42.450 --> 00:09:45.480 could be attributed to that work 143 00:09:45.480 --> 00:09:49.113 and not this sort of special tutorial. 144 00:09:50.610 --> 00:09:53.370 So breaking it up into four groups 145 00:09:53.370 --> 00:09:55.803 isolates the effect of each one. 146 00:09:58.770 --> 00:10:01.560 So the strengths of experiment 147 00:10:01.560 --> 00:10:05.520 is the ability to tightly control things 148 00:10:05.520 --> 00:10:09.360 and carefully measure the effect of things. 149 00:10:09.360 --> 00:10:13.800 They can be fairly low cost and not too time consuming. 150 00:10:13.800 --> 00:10:16.170 It can be done over and over again, 151 00:10:16.170 --> 00:10:18.563 and it has a sort of logical rigor. 152 00:10:22.620 --> 00:10:26.130 The downside is that's our artificiality, 153 00:10:26.130 --> 00:10:28.110 that we really will wanna know 154 00:10:28.110 --> 00:10:31.110 how will someone behave in the real world, 155 00:10:31.110 --> 00:10:36.110 and by doing it in a sort of closely scripted experiment 156 00:10:36.180 --> 00:10:39.753 that may not exactly measure what we want. 157 00:10:42.218 --> 00:10:46.680 So here are the key takeaways of this video lecture. 158 00:10:46.680 --> 00:10:47.513 Thank you.