WEBVTT 1 00:00:01.290 --> 00:00:05.310 Hello, this is the video lecture on survey research 2 00:00:05.310 --> 00:00:09.630 and it's gonna give you information on surveys 3 00:00:09.630 --> 00:00:11.310 and how to do surveys, 4 00:00:11.310 --> 00:00:14.280 as well as looking at different variable types 5 00:00:14.280 --> 00:00:16.830 that you might get from a survey, 6 00:00:16.830 --> 00:00:19.890 and to help you to identify 7 00:00:19.890 --> 00:00:22.050 what kind of variable we have, 8 00:00:22.050 --> 00:00:24.450 because that's a very important factor 9 00:00:24.450 --> 00:00:29.450 in what kind of analysis that we will do with those data, 10 00:00:29.940 --> 00:00:32.703 and something that we'll build on as we go. 11 00:00:36.300 --> 00:00:40.590 So we're gonna look briefly at the idea of a survey 12 00:00:40.590 --> 00:00:45.480 and what sort of topics they can cover. 13 00:00:45.480 --> 00:00:49.500 We'll look at some tips on questionnaire construction, 14 00:00:49.500 --> 00:00:51.690 the strengths and weaknesses of a survey, 15 00:00:51.690 --> 00:00:54.453 and then look at variable types. 16 00:00:56.790 --> 00:01:01.350 So I imagine many, if not all of you, have taken a survey. 17 00:01:01.350 --> 00:01:04.410 It's a very well known method 18 00:01:04.410 --> 00:01:09.410 and the basic definition is that the researcher 19 00:01:09.600 --> 00:01:12.510 designs a standardized set of questions 20 00:01:12.510 --> 00:01:15.750 that are then sent to and responded to 21 00:01:15.750 --> 00:01:18.303 by a number of subjects. 22 00:01:21.540 --> 00:01:24.420 So this is a good way of getting original data 23 00:01:24.420 --> 00:01:26.730 from a fairly large group, 24 00:01:26.730 --> 00:01:28.890 many of the same kind of questions 25 00:01:28.890 --> 00:01:32.970 that you would ask in interviews, behaviors, beliefs, 26 00:01:32.970 --> 00:01:35.340 awareness, attitude, knowledge, 27 00:01:35.340 --> 00:01:37.950 things like that are the types of things 28 00:01:37.950 --> 00:01:40.710 that we ask in social science research. 29 00:01:40.710 --> 00:01:45.710 And because you can both, you get information 30 00:01:49.860 --> 00:01:54.780 from a fairly large group because it tends to be organized 31 00:01:54.780 --> 00:01:59.100 into fairly neat variables and such 32 00:01:59.100 --> 00:02:04.100 that in many cases you can do statistical analysis 33 00:02:05.760 --> 00:02:10.760 and even inference to larger groups in certain cases. 34 00:02:15.870 --> 00:02:20.870 So surveys fall on the right-hand column of this continuum. 35 00:02:25.440 --> 00:02:30.440 This is a chart that you should be pretty familiar with 36 00:02:31.920 --> 00:02:36.920 and know by now, but you tend to get quantitative data 37 00:02:37.050 --> 00:02:39.540 in a nomothetic approach. 38 00:02:39.540 --> 00:02:44.400 You more often ask closed ended questions. 39 00:02:44.400 --> 00:02:48.990 You analyze it through statistics 40 00:02:48.990 --> 00:02:51.090 and they tend to be more reliable. 41 00:02:51.090 --> 00:02:54.750 So this is a slide that you should be pretty familiar 42 00:02:54.750 --> 00:02:59.730 and comfortable with and if you're not, 43 00:02:59.730 --> 00:03:02.010 then we should spend some time in class 44 00:03:02.010 --> 00:03:04.080 making sure that you are, 45 00:03:04.080 --> 00:03:08.673 or you can come to my office hours and we can go over it. 46 00:03:12.030 --> 00:03:16.980 So as we learned in interviews, 47 00:03:16.980 --> 00:03:18.510 there's two main forms, 48 00:03:18.510 --> 00:03:21.750 there's open-ended and there's closed-ended. 49 00:03:21.750 --> 00:03:25.770 Generally speaking, interviews have more open-ended, 50 00:03:25.770 --> 00:03:28.560 tell me about this, what are your experiences? 51 00:03:28.560 --> 00:03:31.680 Whereas surveys tend to be closed-ended 52 00:03:31.680 --> 00:03:36.090 where you have them write in, say a specific number, 53 00:03:36.090 --> 00:03:40.290 like what year were you born or check from a list, 54 00:03:40.290 --> 00:03:43.590 what is your race, what is your gender? 55 00:03:43.590 --> 00:03:46.950 How strongly do you agree? 56 00:03:46.950 --> 00:03:51.950 But for the most part, surveys have closed ended question. 57 00:03:54.060 --> 00:03:58.230 So you get reliable, predictable data. 58 00:03:58.230 --> 00:04:00.780 Each person only has the option 59 00:04:00.780 --> 00:04:04.770 of a certain number of responses 60 00:04:04.770 --> 00:04:09.060 and that makes it easier in certain ways 61 00:04:09.060 --> 00:04:12.240 to analyze them, to count how many said what 62 00:04:12.240 --> 00:04:17.133 and do a sort of correlation across variables. 63 00:04:19.680 --> 00:04:24.680 So again, sometimes you do include an open-ended question 64 00:04:26.070 --> 00:04:29.910 where you let the respondents say what, 65 00:04:29.910 --> 00:04:32.640 put it into their own words. 66 00:04:32.640 --> 00:04:36.570 And again, this sort of veers into qualitative work 67 00:04:36.570 --> 00:04:38.673 and tends to be more valid, 68 00:04:40.560 --> 00:04:45.560 whereas closed ended are more common in surveys. 69 00:04:46.680 --> 00:04:50.250 Probably most of the surveys that you have taken 70 00:04:50.250 --> 00:04:53.610 or administered have more closed ended. 71 00:04:53.610 --> 00:04:58.547 And here you give the respondents a set of options 72 00:05:00.690 --> 00:05:04.263 and they pick one or more that's best for you. 73 00:05:05.430 --> 00:05:09.990 You wanna always sort of have, 74 00:05:09.990 --> 00:05:13.017 either give a complete set of everything 75 00:05:13.017 --> 00:05:18.017 that they might think, or say, give 'em the option of other, 76 00:05:20.550 --> 00:05:25.380 and be clear if you want them to choose one 77 00:05:25.380 --> 00:05:30.380 or choose all that apply to them. 78 00:05:30.420 --> 00:05:33.570 If it is, choose one, 79 00:05:33.570 --> 00:05:37.260 make sure that it's, they're mutually exclusive, 80 00:05:37.260 --> 00:05:40.890 and because they are going to select 81 00:05:40.890 --> 00:05:43.500 from a certain number of answers, 82 00:05:43.500 --> 00:05:47.160 data that you get from closed ended questions 83 00:05:47.160 --> 00:05:49.143 tends to be more reliable. 84 00:05:52.350 --> 00:05:55.660 So here are some considerations 85 00:05:57.090 --> 00:06:00.360 that the questions should be clear, 86 00:06:00.360 --> 00:06:03.750 that just because you understand it, 87 00:06:03.750 --> 00:06:05.520 just because you know what things mean, 88 00:06:05.520 --> 00:06:08.730 that doesn't mean the respondents do, 89 00:06:08.730 --> 00:06:11.130 you may need to define things. 90 00:06:11.130 --> 00:06:16.130 You wanna avoid double barrelled questions such as 91 00:06:17.400 --> 00:06:21.063 should we cut defense spending and fund education? 92 00:06:22.350 --> 00:06:24.930 Maybe some want to do one or the other, 93 00:06:24.930 --> 00:06:27.570 cut defense, but not for fund education, 94 00:06:27.570 --> 00:06:30.780 not cut the defense but do fund education 95 00:06:30.780 --> 00:06:35.310 so split that into two different questions. 96 00:06:35.310 --> 00:06:40.310 You don't wanna lead on, sort of have your opinion, 97 00:06:43.666 --> 00:06:48.666 you know, is CDAE the best department in the world 98 00:06:51.360 --> 00:06:54.450 or things like that, where you know 99 00:06:54.450 --> 00:06:57.540 it's clear where you're sort of going with this, 100 00:06:57.540 --> 00:06:59.760 you want them to be much more neutral 101 00:06:59.760 --> 00:07:04.760 and it's absolutely imperative that your respondent 102 00:07:06.810 --> 00:07:09.180 is able to answer the question, 103 00:07:09.180 --> 00:07:13.200 if not, then it's a completely useless exercise. 104 00:07:13.200 --> 00:07:16.800 So they have to understand the question 105 00:07:16.800 --> 00:07:20.370 and they have to know what the right answer is for them 106 00:07:20.370 --> 00:07:24.330 and they must be able to respond 107 00:07:24.330 --> 00:07:27.153 and sort of pick that right answer for them. 108 00:07:32.550 --> 00:07:35.670 Sometimes you may need a I don't know option 109 00:07:35.670 --> 00:07:37.743 and that can be an important thing. 110 00:07:38.730 --> 00:07:43.730 You also want the survey to be as short as it can be. 111 00:07:44.520 --> 00:07:47.430 And so you want the the respondent 112 00:07:47.430 --> 00:07:51.720 to be able to fairly quickly read and answer, 113 00:07:51.720 --> 00:07:55.470 read, know what the right response is for them 114 00:07:55.470 --> 00:07:59.050 and choose that and not get bogged down 115 00:08:00.150 --> 00:08:02.160 taking a very long survey 116 00:08:02.160 --> 00:08:05.580 because in many cases if they get frustrated 117 00:08:05.580 --> 00:08:10.323 they will just stop and you won't get a full set of answers. 118 00:08:12.180 --> 00:08:17.180 So we talked about this before, these top three 119 00:08:17.490 --> 00:08:21.420 are ones that we've already talked about, 120 00:08:21.420 --> 00:08:25.950 general to specific, more important and safe to risky. 121 00:08:25.950 --> 00:08:30.950 I prefer to put the demographics last. 122 00:08:32.850 --> 00:08:37.850 Sometimes folks say at first because they are pretty easy 123 00:08:38.550 --> 00:08:43.053 and they draw you in, but I think it's better to do it last. 124 00:08:47.736 --> 00:08:51.870 The reason last, is if you ask 125 00:08:51.870 --> 00:08:55.380 a bunch of demographic questions first, 126 00:08:55.380 --> 00:08:58.170 especially things that sort of have to do 127 00:08:58.170 --> 00:09:02.160 with their identity, maybe race, ethnicity, 128 00:09:02.160 --> 00:09:06.600 religion, gender, gender identity, things like that, 129 00:09:06.600 --> 00:09:08.760 then it may be that that is going 130 00:09:08.760 --> 00:09:12.540 to sort of flavor their responses, 131 00:09:12.540 --> 00:09:15.930 and they will think well, this is how someone 132 00:09:15.930 --> 00:09:20.930 of my race, of my gender et cetera, 133 00:09:21.210 --> 00:09:23.580 should be answering that. 134 00:09:23.580 --> 00:09:27.033 So that is why I tend to put them last. 135 00:09:31.650 --> 00:09:35.730 Have clear instructions, like, put an X 136 00:09:35.730 --> 00:09:39.210 or click on this, et cetera. 137 00:09:39.210 --> 00:09:42.690 It's good to break it up into subsections. 138 00:09:42.690 --> 00:09:46.170 So sort of logical parts so they know sort of where they are 139 00:09:46.170 --> 00:09:47.970 and where they're going, 140 00:09:47.970 --> 00:09:51.060 and make sure that there are clear instruction. 141 00:09:51.060 --> 00:09:56.040 Do you want them to only choose one versus all that apply, 142 00:09:56.040 --> 00:09:59.580 if you rank sort of how well you do that or rating, 143 00:09:59.580 --> 00:10:02.733 but just make sure that the instructions are very clear, 144 00:10:05.460 --> 00:10:08.673 it's very important to pilot surveys. 145 00:10:09.720 --> 00:10:12.570 Even if you think there are no errors, 146 00:10:12.570 --> 00:10:15.450 it's very clear everything makes sense, 147 00:10:15.450 --> 00:10:18.660 if you send it to folks and have them take it, 148 00:10:18.660 --> 00:10:23.310 and send you feedback, they, in my experience, 149 00:10:23.310 --> 00:10:25.230 they almost always find things wrong 150 00:10:25.230 --> 00:10:28.143 that you can then fix and that's a good thing. 151 00:10:29.430 --> 00:10:32.520 So it's good to do something, 152 00:10:32.520 --> 00:10:37.520 like give it to 10 people who aren't your actual respondents 153 00:10:39.210 --> 00:10:43.080 and you can ask 'em three things. 154 00:10:43.080 --> 00:10:45.780 How long did it take you? 155 00:10:45.780 --> 00:10:47.640 Because if it took you a lot longer, 156 00:10:47.640 --> 00:10:50.343 it took them a lot longer than you thought, 157 00:10:52.410 --> 00:10:55.560 again maybe become annoyed or demoralized 158 00:10:55.560 --> 00:10:59.640 and not finish it. 159 00:10:59.640 --> 00:11:02.400 Ask if there was anything unclear, 160 00:11:02.400 --> 00:11:05.280 and then ask if they saw any errors, 161 00:11:05.280 --> 00:11:08.553 any typos or spelling errors or anything like that. 162 00:11:11.670 --> 00:11:15.813 All else, equals shorter is better, 163 00:11:16.710 --> 00:11:21.033 15 to 20 minutes is probably the max, 164 00:11:22.440 --> 00:11:25.530 and like get the information that you need 165 00:11:25.530 --> 00:11:27.930 but don't ask things that you don't need, 166 00:11:27.930 --> 00:11:29.550 because again it makes it longer, 167 00:11:29.550 --> 00:11:33.693 and there's the risk that they will stop taking it. 168 00:11:36.210 --> 00:11:38.880 So let's move to what are the strengths 169 00:11:38.880 --> 00:11:40.740 and weaknesses of a survey? 170 00:11:40.740 --> 00:11:44.400 So you get a larger number of responses 171 00:11:44.400 --> 00:11:48.870 and in many ways the analysis is more straightforward. 172 00:11:48.870 --> 00:11:52.290 You can look at frequencies and means things like that. 173 00:11:52.290 --> 00:11:55.740 You can look at correlations, 174 00:11:55.740 --> 00:11:58.380 and it's much more straightforward, 175 00:11:58.380 --> 00:12:00.723 and you get much more reliable data. 176 00:12:03.570 --> 00:12:07.050 The weaknesses are a lack of depth. 177 00:12:07.050 --> 00:12:11.490 So if you choose, say choose from one of these five things, 178 00:12:11.490 --> 00:12:13.470 maybe none of those five things 179 00:12:13.470 --> 00:12:16.457 are really exactly what they think or feel, 180 00:12:16.457 --> 00:12:21.457 so it may be less valid and you might miss things. 181 00:12:21.690 --> 00:12:25.440 And last, it's hard to change things in midstream 182 00:12:25.440 --> 00:12:29.070 that with an interview you can in some cases go back 183 00:12:29.070 --> 00:12:32.790 and say, hey there's this a new question we have 184 00:12:32.790 --> 00:12:35.250 based on what we learned that we know is important 185 00:12:35.250 --> 00:12:38.280 and we want to ask it to you now, 186 00:12:38.280 --> 00:12:42.573 much harder to add questions on a survey like that. 187 00:12:46.680 --> 00:12:50.340 So now let's talk about the four types of variables 188 00:12:50.340 --> 00:12:53.160 that you can get from a survey 189 00:12:53.160 --> 00:12:57.000 or four types of variables in general, 190 00:12:57.000 --> 00:13:01.350 and they are nominal, ordinal, interval and ratio. 191 00:13:01.350 --> 00:13:04.800 Note that an interval is the rarest kind. 192 00:13:04.800 --> 00:13:09.570 It's not really one that you and encounter much, 193 00:13:09.570 --> 00:13:12.780 but they do exist and I will mention them, 194 00:13:12.780 --> 00:13:16.380 but the other kinds are much more common 195 00:13:16.380 --> 00:13:19.650 and it's very important that you are able 196 00:13:19.650 --> 00:13:24.570 to identify what kind of variable is this 197 00:13:24.570 --> 00:13:27.120 so that you can do the right analysis. 198 00:13:27.120 --> 00:13:29.820 So we're really going to be building on this part 199 00:13:29.820 --> 00:13:32.160 for much of the rest of the class 200 00:13:32.160 --> 00:13:34.110 and it's important that you understand. 201 00:13:36.540 --> 00:13:38.280 So we'll start with nominal, 202 00:13:38.280 --> 00:13:41.520 these are also called categorical, 203 00:13:41.520 --> 00:13:43.140 and you're just naming things. 204 00:13:43.140 --> 00:13:46.320 So note that there's no higher or lower, 205 00:13:46.320 --> 00:13:49.803 there's no more or less, there's no ranking. 206 00:13:50.910 --> 00:13:54.450 And these are things like you know, nominal, 207 00:13:54.450 --> 00:13:56.850 like, they only have names. 208 00:13:56.850 --> 00:13:59.970 So if you say what state were you born in? 209 00:13:59.970 --> 00:14:04.380 It's not like one state is is higher or lower 210 00:14:04.380 --> 00:14:06.450 or et cetera. 211 00:14:06.450 --> 00:14:08.283 What's your gender, 212 00:14:09.780 --> 00:14:13.560 What's your eye color, your favorite sports team, 213 00:14:13.560 --> 00:14:18.390 major et cetera, that these have no sort of ranking, 214 00:14:18.390 --> 00:14:21.780 there's no numerical data, it's just pick one 215 00:14:21.780 --> 00:14:22.920 that's right for you, 216 00:14:22.920 --> 00:14:26.013 or pick one or more that's right for you. 217 00:14:29.310 --> 00:14:32.790 Ordinal as you might guess have order. 218 00:14:32.790 --> 00:14:37.790 So here this tends to be you're either ranking or rating, 219 00:14:38.820 --> 00:14:43.820 so you can, you know that one is higher 220 00:14:45.330 --> 00:14:50.330 or lower than another, but you can't tell 221 00:14:50.790 --> 00:14:52.830 how far apart they are. 222 00:14:52.830 --> 00:14:54.300 So let's say that you ask 223 00:14:54.300 --> 00:14:56.760 what's your favorite ice cream flavor? 224 00:14:56.760 --> 00:14:58.650 And someone says one, chocolate, 225 00:14:58.650 --> 00:15:03.210 two, strawberry, three, vanilla. 226 00:15:03.210 --> 00:15:06.840 So you know that they like chocolate more than strawberry, 227 00:15:06.840 --> 00:15:09.390 and they like strawberry more than vanilla, 228 00:15:09.390 --> 00:15:13.290 and you also know they like chocolate more than vanilla, 229 00:15:13.290 --> 00:15:15.720 but two things that you don't know 230 00:15:15.720 --> 00:15:19.500 are how far apart are they? 231 00:15:19.500 --> 00:15:23.940 So you might really love chocolate 232 00:15:23.940 --> 00:15:28.590 and then you only like strawberry 233 00:15:28.590 --> 00:15:31.923 a little bit more than vanilla. 234 00:15:33.090 --> 00:15:36.990 We often do as well a rating on a scale. 235 00:15:36.990 --> 00:15:40.500 How do you agree, how often, things like this. 236 00:15:40.500 --> 00:15:44.680 And in that same way you don't know how far apart 237 00:15:44.680 --> 00:15:48.090 four and five are, and your strongly agree 238 00:15:48.090 --> 00:15:52.980 and my strongly agree aren't necessarily the same. 239 00:15:52.980 --> 00:15:55.863 You just know that I agree with, 240 00:15:58.350 --> 00:16:02.043 if I say it's a 5, I strongly agree, 241 00:16:02.910 --> 00:16:07.910 I agree with it more than if I had said 4, 3, 2 or 1. 242 00:16:09.060 --> 00:16:14.060 And note that neither nominal nor ordinal have any units 243 00:16:15.450 --> 00:16:18.393 and we're gonna see that the next two do. 244 00:16:22.230 --> 00:16:27.230 So an interval is you can tell how far apart they are 245 00:16:27.810 --> 00:16:30.870 and note that these have units. 246 00:16:30.870 --> 00:16:35.520 Two very well known ones are degrees fahrenheit 247 00:16:35.520 --> 00:16:39.120 or degrees celsius and year of birth. 248 00:16:39.120 --> 00:16:42.630 So if it's for degrees, 249 00:16:42.630 --> 00:16:47.630 if it was 20 degrees today and 40 yesterday, 250 00:16:49.110 --> 00:16:52.620 you can say it's 20 degrees cooler, 251 00:16:52.620 --> 00:16:56.370 but you can't say it's twice as cold or twice as warm, 252 00:16:56.370 --> 00:16:59.190 same as year of birth. 253 00:16:59.190 --> 00:17:04.190 So degrees is a unit, year is a unit. 254 00:17:07.170 --> 00:17:09.600 But you can't say that someone 255 00:17:09.600 --> 00:17:14.600 who was born in the year 1000 256 00:17:18.090 --> 00:17:21.240 is twice as old as the person 257 00:17:21.240 --> 00:17:24.630 or as young as the person born in the year 2000, 258 00:17:24.630 --> 00:17:28.203 it sort of doesn't make sense in that way. 259 00:17:29.280 --> 00:17:31.740 Ratio you can express. 260 00:17:31.740 --> 00:17:36.740 So ratio not only has units, it has a true zero. 261 00:17:38.190 --> 00:17:40.470 So if we look at how far apart 262 00:17:40.470 --> 00:17:45.450 are Philly and Boston from Burlington 263 00:17:45.450 --> 00:17:49.290 that you can say not only, 264 00:17:49.290 --> 00:17:53.670 you could express it as an interval, it's 400 miles more, 265 00:17:53.670 --> 00:17:57.810 but it's also a ratio 2.85 times farther. 266 00:18:02.340 --> 00:18:07.290 So a ratio, so it has units. 267 00:18:07.290 --> 00:18:12.290 So in these examples, it's miles, dollars and points. 268 00:18:13.260 --> 00:18:17.010 And not only can you tell how far apart they are, 269 00:18:17.010 --> 00:18:19.260 but you can express it as a ratio 270 00:18:19.260 --> 00:18:22.830 and say this is twice as big or five times as, 271 00:18:22.830 --> 00:18:26.370 or 3.12 times, et cetera. 272 00:18:26.370 --> 00:18:30.150 So it can be expressed as a ratio. 273 00:18:30.150 --> 00:18:33.300 And note that all these things have true zeros, 274 00:18:33.300 --> 00:18:36.390 something could could be zero miles away, 275 00:18:36.390 --> 00:18:41.390 someone might earn $0, and a team might allow zero points. 276 00:18:42.690 --> 00:18:46.570 At least theoretically, these things make sense 277 00:18:49.260 --> 00:18:54.260 whereas born in the year zero really does not make sense. 278 00:18:57.150 --> 00:18:59.850 So here is a recap. 279 00:18:59.850 --> 00:19:02.220 Again, make sure that you understand these, 280 00:19:02.220 --> 00:19:05.340 and know what they mean, 281 00:19:05.340 --> 00:19:09.330 and are able to identify and name them. 282 00:19:09.330 --> 00:19:11.940 So if I would say on an exam, 283 00:19:11.940 --> 00:19:14.700 look at this, what kind of variable is it? 284 00:19:14.700 --> 00:19:17.400 That's something that you should know how to do. 285 00:19:17.400 --> 00:19:20.305 If I would say give me an example 286 00:19:20.305 --> 00:19:22.440 of an ordinal variable, 287 00:19:22.440 --> 00:19:24.840 that's something that you should know how to do. 288 00:19:26.100 --> 00:19:29.970 So make sure that you do and talk to me if you can't, 289 00:19:29.970 --> 00:19:32.070 and we'll get you through it 290 00:19:32.070 --> 00:19:35.190 'cause that's gonna be a really important task 291 00:19:35.190 --> 00:19:37.530 and a bit of knowledge that you'll need 292 00:19:37.530 --> 00:19:42.530 both for the exams and for the class project, 293 00:19:45.000 --> 00:19:50.000 when we do our survey and we analyze the data. 294 00:19:52.140 --> 00:19:53.343 That is what we did. 295 00:19:57.690 --> 00:20:02.583 Here are the key things and thank you for your time.