1 00:00:03,210 --> 00:00:05,190 [Instructor] Hello, everyone. 2 00:00:05,190 --> 00:00:09,150 And welcome to the first recorded lecture 3 00:00:09,150 --> 00:00:12,660 of CDAE 250 for this fall. 4 00:00:12,660 --> 00:00:15,660 And this class, we're gonna be talking about 5 00:00:15,660 --> 00:00:17,520 how do we know things 6 00:00:17,520 --> 00:00:22,274 and some basic research terminology and vocabulary 7 00:00:22,274 --> 00:00:25,863 that we'll be using throughout the semester. 8 00:00:30,900 --> 00:00:32,880 So what is social science research? 9 00:00:32,880 --> 00:00:36,330 That's what we'll be doing this semester. 10 00:00:36,330 --> 00:00:38,710 And this quotation talks about 11 00:00:39,870 --> 00:00:43,653 how it is the study of human behavior. 12 00:00:45,150 --> 00:00:49,920 And I want to emphasize that it is more 13 00:00:49,920 --> 00:00:53,250 the study of groups of individuals. 14 00:00:53,250 --> 00:00:58,250 It's the study of social units and groups 15 00:01:01,230 --> 00:01:06,150 more than it is the study of individual behavior, 16 00:01:06,150 --> 00:01:08,506 which is more like psychology, 17 00:01:08,506 --> 00:01:12,655 but the idea being to understand patterns of behavior, 18 00:01:12,655 --> 00:01:17,655 norms, rules, expectations, behaviors of groups. 19 00:01:20,730 --> 00:01:24,150 That tends to be the kind of topics 20 00:01:24,150 --> 00:01:26,823 that social science addresses. 21 00:01:32,370 --> 00:01:35,280 So if we're gonna understand things, 22 00:01:35,280 --> 00:01:38,310 hopefully, we do so correctly. 23 00:01:38,310 --> 00:01:40,890 And how do we know things? 24 00:01:40,890 --> 00:01:42,240 How do we know things are true? 25 00:01:42,240 --> 00:01:44,160 How do we learn things? 26 00:01:44,160 --> 00:01:49,160 And how do we know what is credible? 27 00:01:49,350 --> 00:01:51,150 I think it's fair to say 28 00:01:51,150 --> 00:01:56,150 that there's a lot of misinformation, half truths, 29 00:01:56,280 --> 00:01:57,799 outright lies being told 30 00:01:57,799 --> 00:02:01,230 out in the the world all the time. 31 00:02:01,230 --> 00:02:03,510 Sometimes the person who tells you 32 00:02:03,510 --> 00:02:06,630 this believes it true, sometimes not. 33 00:02:06,630 --> 00:02:07,860 So how do we know? 34 00:02:07,860 --> 00:02:10,004 How do we know what is credible? 35 00:02:10,004 --> 00:02:14,220 That's a really big objective of mine for all of you, 36 00:02:14,220 --> 00:02:17,760 is to walk out of this class really having 37 00:02:17,760 --> 00:02:20,064 a very finely-honed, 38 00:02:20,064 --> 00:02:23,760 as I said in the first class, BS detector, 39 00:02:23,760 --> 00:02:28,230 that you can really have the skills and knowledge 40 00:02:28,230 --> 00:02:33,157 to be able to judge the credibility of statements 41 00:02:33,157 --> 00:02:36,690 and of research. 42 00:02:36,690 --> 00:02:41,520 And part of knowing what's credible 43 00:02:41,520 --> 00:02:44,311 is to know what mistakes do we make 44 00:02:44,311 --> 00:02:47,223 and why do we make them? 45 00:02:52,350 --> 00:02:55,898 One way of thinking about how we know things 46 00:02:55,898 --> 00:03:00,330 is to look at these two options. 47 00:03:00,330 --> 00:03:03,420 Sometimes we know things from our own experience, 48 00:03:03,420 --> 00:03:07,440 sometimes we know things from tradition or authority. 49 00:03:07,440 --> 00:03:12,440 And in each case, this is not social science research. 50 00:03:13,650 --> 00:03:17,910 Neither of these is sort of a rigorous process 51 00:03:17,910 --> 00:03:22,650 that we go through to discover information and meaning. 52 00:03:22,650 --> 00:03:24,247 So there are things that you've learned 53 00:03:24,247 --> 00:03:28,502 about UVM for your own experience. 54 00:03:28,502 --> 00:03:32,280 I assume that all of you have been 55 00:03:32,280 --> 00:03:34,680 in face-to-face classrooms, 56 00:03:34,680 --> 00:03:38,610 and the very first time you went to that classroom, 57 00:03:38,610 --> 00:03:41,940 it might have taken you a bit of time, 58 00:03:41,940 --> 00:03:44,430 but soon after you've been there 59 00:03:44,430 --> 00:03:46,887 and probably by the second or third week 60 00:03:46,887 --> 00:03:50,280 through your own experience going there 61 00:03:50,280 --> 00:03:53,340 that you really don't even have to think about it anymore, 62 00:03:53,340 --> 00:03:56,550 that you know where the classroom is, 63 00:03:56,550 --> 00:03:59,793 where class meets, and you just do it. 64 00:04:00,660 --> 00:04:01,950 There are also many things 65 00:04:01,950 --> 00:04:05,880 that we learn by tradition or authority. 66 00:04:05,880 --> 00:04:08,340 So when I think about tradition, 67 00:04:08,340 --> 00:04:11,760 maybe one really simple way 68 00:04:11,760 --> 00:04:15,323 of thinking about this is cultural norms. 69 00:04:18,570 --> 00:04:21,660 So there are certain behaviors 70 00:04:21,660 --> 00:04:25,540 that when you're in a face-to-face classroom 71 00:04:26,910 --> 00:04:30,300 that are acceptable and that are not acceptable. 72 00:04:30,300 --> 00:04:35,300 If you sit in a chair, write notes, 73 00:04:36,930 --> 00:04:40,238 sit fairly quietly and still, 74 00:04:40,238 --> 00:04:44,169 tradition says that that's sort of what the expected 75 00:04:44,169 --> 00:04:48,360 and acceptable behavior of a student is. 76 00:04:48,360 --> 00:04:50,370 If you would stand up and dance, 77 00:04:50,370 --> 00:04:54,900 stand on your seat, turn around wildly, 78 00:04:54,900 --> 00:04:59,900 tap your fellow students on the shoulder, 79 00:05:00,120 --> 00:05:05,120 bump them, do all sort of distracting things like that, 80 00:05:06,960 --> 00:05:10,980 it's doubtful that you learn that by your own experience. 81 00:05:10,980 --> 00:05:13,920 It's doubtful that you ever read a manual 82 00:05:13,920 --> 00:05:16,189 that said don't do those things. 83 00:05:16,189 --> 00:05:20,400 You just sort of learned it through tradition. 84 00:05:20,400 --> 00:05:25,400 In the same way, much of what we learn is by authority. 85 00:05:27,150 --> 00:05:30,330 And frankly, in this class, 86 00:05:30,330 --> 00:05:34,410 I am an authority on this subject matter. 87 00:05:34,410 --> 00:05:37,057 I hold a PhD. 88 00:05:39,900 --> 00:05:44,900 I have 10 years of experience teaching this class. 89 00:05:44,970 --> 00:05:49,320 I have almost 20 years of experience 90 00:05:49,320 --> 00:05:51,720 doing social science research. 91 00:05:51,720 --> 00:05:55,740 So I have earned the right, hopefully, 92 00:05:55,740 --> 00:06:00,740 in your mind to be credible when I tell you 93 00:06:02,640 --> 00:06:07,050 what are effective and credible methods 94 00:06:07,050 --> 00:06:11,073 for carrying out social science research. 95 00:06:13,410 --> 00:06:18,393 But even beyond this, sometimes we make mistakes. 96 00:06:25,050 --> 00:06:28,620 Now I'm going to walk you through some of the common errors 97 00:06:28,620 --> 00:06:32,859 that we make both in our everyday lives 98 00:06:32,859 --> 00:06:36,633 and in conducting research. 99 00:06:41,280 --> 00:06:44,970 The three of them, and I'm going to go through each one, 100 00:06:44,970 --> 00:06:48,510 are inaccurate observation, 101 00:06:48,510 --> 00:06:53,510 selective observation, and illogical reasoning. 102 00:06:53,790 --> 00:06:57,723 And we humans are prone to all three of these things. 103 00:07:02,790 --> 00:07:06,093 The first of these is inaccurate observation. 104 00:07:07,890 --> 00:07:12,180 That we very often just fail to pay attention. 105 00:07:12,180 --> 00:07:16,140 That sometimes the details of things 106 00:07:16,140 --> 00:07:18,390 simply are not important. 107 00:07:18,390 --> 00:07:20,550 So if I were to ask you, 108 00:07:20,550 --> 00:07:24,560 assuming that at some point during the day, yesterday, 109 00:07:25,830 --> 00:07:27,390 you were out and about, 110 00:07:27,390 --> 00:07:31,653 you were walking around town, doing things like that, 111 00:07:33,330 --> 00:07:37,950 I could ask you, how many blue cars did you pass? 112 00:07:37,950 --> 00:07:42,950 How many blue cars did you see while you were outside? 113 00:07:44,400 --> 00:07:46,890 Since this wasn't an important thing, 114 00:07:46,890 --> 00:07:50,940 not anything that you really had your mind on, 115 00:07:50,940 --> 00:07:52,770 I doubt if any of you 116 00:07:52,770 --> 00:07:56,400 could give me anything approaching 117 00:07:56,400 --> 00:07:59,292 an accurate estimate even. 118 00:07:59,292 --> 00:08:02,430 But if I asked you, tomorrow, 119 00:08:02,430 --> 00:08:06,540 go outside and count how many blue cars 120 00:08:06,540 --> 00:08:09,480 that you see during the the day. 121 00:08:09,480 --> 00:08:11,280 I'm not asking you to do this. 122 00:08:11,280 --> 00:08:13,140 This is just an example. 123 00:08:13,140 --> 00:08:16,440 I think that you would by paying attention, 124 00:08:16,440 --> 00:08:18,570 and if I was grading you on it, 125 00:08:18,570 --> 00:08:20,340 you would think it was important, 126 00:08:20,340 --> 00:08:24,533 and you would have a much more accurate number 127 00:08:25,920 --> 00:08:28,890 of the blue cars that you passed. 128 00:08:28,890 --> 00:08:30,197 But many things in life, 129 00:08:30,197 --> 00:08:33,180 we just sort of don't pay attention to 'em 130 00:08:33,180 --> 00:08:35,613 because they don't seem important at the time. 131 00:08:41,580 --> 00:08:46,290 One of the most common one is selective observation. 132 00:08:46,290 --> 00:08:50,790 We humans are very good at seeing patterns, 133 00:08:50,790 --> 00:08:54,183 and that has served us well throughout our evolution. 134 00:08:55,230 --> 00:08:59,790 However, we tend to look for events 135 00:08:59,790 --> 00:09:00,930 that fit a pattern, 136 00:09:00,930 --> 00:09:03,480 that once we have established a pattern, 137 00:09:03,480 --> 00:09:08,480 we're much more likely to see things that fit that pattern. 138 00:09:10,860 --> 00:09:14,850 So the idea of if all my life, 139 00:09:14,850 --> 00:09:18,270 all I've ever seen were white swans, 140 00:09:18,270 --> 00:09:21,660 that I might conclude that all swans are white. 141 00:09:21,660 --> 00:09:23,610 And if I see a white swan, I'm like, 142 00:09:23,610 --> 00:09:25,941 see, look, there's a white swan. 143 00:09:25,941 --> 00:09:28,901 There's a swan, it's white. 144 00:09:28,901 --> 00:09:32,580 That means all swans are white. 145 00:09:32,580 --> 00:09:34,635 And this was actually an example 146 00:09:34,635 --> 00:09:38,880 that was developed by a philosopher 147 00:09:38,880 --> 00:09:41,034 of science named Gunnar Myrdal 148 00:09:41,034 --> 00:09:43,585 who used this black and white swan thing. 149 00:09:43,585 --> 00:09:46,230 But we tend to do that, 150 00:09:46,230 --> 00:09:50,070 that we tend to only notice things 151 00:09:50,070 --> 00:09:52,650 that fit our patterns and we discount things 152 00:09:52,650 --> 00:09:54,326 that don't fit our pattern. 153 00:09:54,326 --> 00:09:57,420 And when we see something that fits the pattern 154 00:09:57,420 --> 00:10:02,010 that we have, we say, aha, see, I'm right. 155 00:10:02,010 --> 00:10:06,360 It's a very natural common way that our brains work. 156 00:10:06,360 --> 00:10:09,512 But it's one thing that we really have to be aware of 157 00:10:09,512 --> 00:10:13,927 and cautious of in social science research. 158 00:10:13,927 --> 00:10:18,927 And this is very closely related to a very common bias 159 00:10:20,490 --> 00:10:25,140 that we all have, social scientists and not, 160 00:10:25,140 --> 00:10:27,027 called confirmation bias. 161 00:10:27,027 --> 00:10:29,613 And I'm gonna speak to you a bit about that now. 162 00:10:32,430 --> 00:10:37,430 So confirmation bias is that we give preferential treatment 163 00:10:38,880 --> 00:10:43,743 to evidence that supports our preexisting belief, 164 00:10:45,150 --> 00:10:50,150 and we tend to look for cases or evidence or examples 165 00:10:52,050 --> 00:10:54,840 that fit what we already think, 166 00:10:54,840 --> 00:10:57,690 and we tend to overweigh them, 167 00:10:57,690 --> 00:10:59,670 that we put a lot of credibility 168 00:10:59,670 --> 00:11:03,180 in things that we think are true, 169 00:11:03,180 --> 00:11:06,810 and we tend to easily discount and dismiss things 170 00:11:06,810 --> 00:11:10,500 that don't fit into this pattern. 171 00:11:10,500 --> 00:11:11,333 And I think it's obvious 172 00:11:11,333 --> 00:11:16,333 in today's highly politicized environment 173 00:11:16,407 --> 00:11:21,407 that if you were to talk to a Donald Trump supporter 174 00:11:22,890 --> 00:11:25,110 who watches Fox News, 175 00:11:25,110 --> 00:11:27,536 and you show them something that shows 176 00:11:27,536 --> 00:11:30,810 that Donald Trump has done a very good job, 177 00:11:30,810 --> 00:11:33,780 that you would be like, yes, see, it's true. 178 00:11:33,780 --> 00:11:35,040 This is true. 179 00:11:35,040 --> 00:11:36,840 This fits what I'm saying. 180 00:11:36,840 --> 00:11:40,920 And this proves, see, Donald Trump is doing a great job. 181 00:11:40,920 --> 00:11:44,490 Whereas if you're more on the left, 182 00:11:44,490 --> 00:11:49,125 if you are not a fan of President Trump, 183 00:11:49,125 --> 00:11:54,125 and you oppose much or all of what he does, you will say, 184 00:11:55,650 --> 00:11:58,020 well, this, you know, 185 00:11:58,020 --> 00:12:01,425 this really isn't true or this isn't important 186 00:12:01,425 --> 00:12:04,500 or this, you know, it's true, 187 00:12:04,500 --> 00:12:09,500 but there's all these other reasons that counteract this. 188 00:12:09,901 --> 00:12:14,901 That you will tend to dismiss some bit of evidence 189 00:12:15,457 --> 00:12:20,370 that claims that Donald Trump is doing a good job. 190 00:12:20,370 --> 00:12:24,510 And we are very, very prone to this. 191 00:12:24,510 --> 00:12:25,440 We all are. 192 00:12:25,440 --> 00:12:28,650 Don't feel like it makes us bad or anything like that. 193 00:12:28,650 --> 00:12:33,650 We all do, but it's very important that we are aware of it. 194 00:12:40,560 --> 00:12:45,560 So here is a paragraph with the URL 195 00:12:47,847 --> 00:12:52,143 if you would like to know more about confirmation bias, 196 00:12:53,398 --> 00:12:57,523 and that is that we seek out things that we seek out, 197 00:12:58,425 --> 00:13:02,640 and we put credibility into things 198 00:13:02,640 --> 00:13:05,361 that support what we already think, 199 00:13:05,361 --> 00:13:10,361 and we dismiss or overlook things 200 00:13:11,040 --> 00:13:16,040 that do not conform to our preexisting beliefs. 201 00:13:17,370 --> 00:13:22,110 And in a few minutes and during our discussion in class, 202 00:13:22,110 --> 00:13:25,320 I wanna dive into why this could be a problem. 203 00:13:25,320 --> 00:13:26,880 So hold that thought. 204 00:13:26,880 --> 00:13:31,880 You're gonna see confirmation bias again, hint, hint. 205 00:13:36,720 --> 00:13:41,720 Another common error is illogical reasoning 206 00:13:42,420 --> 00:13:45,247 like the so-called gambler's fallacy. 207 00:13:45,247 --> 00:13:50,190 That if you've lost five straight hands, 208 00:13:50,190 --> 00:13:52,273 and you're almost out of money 209 00:13:52,273 --> 00:13:54,060 that you think, well, 210 00:13:54,060 --> 00:13:58,950 the odds are that I'm going to win this time. 211 00:13:58,950 --> 00:14:03,090 That you sort of skew the odds to think that, well, 212 00:14:03,090 --> 00:14:05,180 I've lost so many straight times, 213 00:14:05,180 --> 00:14:09,393 now it's my turn, and I should make a big bet here. 214 00:14:10,230 --> 00:14:15,230 Whereas in almost any sort of gambling scenario, 215 00:14:18,660 --> 00:14:20,910 the odds are what they are, 216 00:14:20,910 --> 00:14:24,480 that they don't change whether you have won 217 00:14:24,480 --> 00:14:27,453 a whole bunch in a row or lost a whole bunch in a row. 218 00:14:28,500 --> 00:14:32,917 Your odds of winning lottery ticket do not have anything 219 00:14:32,917 --> 00:14:37,917 to do with whether you have bought a whole lot of ones 220 00:14:38,460 --> 00:14:42,660 that won or a whole bunch of ones that lost or a mixture. 221 00:14:42,660 --> 00:14:44,460 The odds don't change. 222 00:14:44,460 --> 00:14:49,460 In the same way, in a more simple example, 223 00:14:50,236 --> 00:14:53,850 if you take a fair coin and flip it 10 times 224 00:14:53,850 --> 00:14:58,200 and every single time that its heads, you might think, 225 00:14:58,200 --> 00:15:03,030 well, it's been 10 in a row heads, I'm due for a tails now. 226 00:15:03,030 --> 00:15:06,750 But the actual probability of heads 227 00:15:06,750 --> 00:15:11,740 or tails on that particular flip is still 50/50. 228 00:15:18,180 --> 00:15:19,890 I talked about the problems. 229 00:15:19,890 --> 00:15:22,590 Now I'm going to talk about the solutions. 230 00:15:22,590 --> 00:15:26,430 What can we as social scientists do 231 00:15:26,430 --> 00:15:30,633 to overcome these common errors? 232 00:15:31,980 --> 00:15:34,650 So first is intent. 233 00:15:34,650 --> 00:15:38,310 Much of the intent, as I said, count blue cars, 234 00:15:38,310 --> 00:15:40,693 to specifically pay attention, 235 00:15:40,693 --> 00:15:45,693 to really do a good job of paying attention 236 00:15:45,930 --> 00:15:50,930 and to have the intention of doing the next two points here. 237 00:15:52,800 --> 00:15:57,800 First is to use recording devices, to take notes as we go. 238 00:15:58,650 --> 00:16:01,710 So if you were counting blue cars, 239 00:16:01,710 --> 00:16:03,759 you might wanna bring a tablet 240 00:16:03,759 --> 00:16:08,759 and a piece of paper and just make slashes as you go. 241 00:16:09,750 --> 00:16:14,750 In many cases, when you're doing interviews, 242 00:16:15,390 --> 00:16:20,390 you may wish to audio or even video record the subject, 243 00:16:21,780 --> 00:16:25,500 or at very least, take notes as you go, 244 00:16:25,500 --> 00:16:27,354 or maybe all three, 245 00:16:27,354 --> 00:16:32,354 so that, you know, much less is lost. 246 00:16:35,010 --> 00:16:38,973 It's also important to look for deviant cases. 247 00:16:43,200 --> 00:16:46,950 Maybe the example of you wanna know 248 00:16:46,950 --> 00:16:51,950 what the people in a certain area think about a policy. 249 00:16:53,700 --> 00:16:56,550 Maybe what do students think 250 00:16:56,550 --> 00:17:00,330 about the reopening of the university? 251 00:17:00,330 --> 00:17:05,330 So if you go and you keep talking to the same people 252 00:17:05,460 --> 00:17:08,280 and hearing the same thing, 253 00:17:08,280 --> 00:17:11,310 you might be getting into that all swans are white thing. 254 00:17:11,310 --> 00:17:13,445 So you may wanna think about, well, 255 00:17:13,445 --> 00:17:14,908 who is it out there 256 00:17:14,908 --> 00:17:18,240 that might have a different point of view? 257 00:17:18,240 --> 00:17:23,240 Seek out that person, hear their point of view. 258 00:17:23,640 --> 00:17:25,522 Sort of seek out this deviant case 259 00:17:25,522 --> 00:17:28,350 that doesn't fit in with what you've been hearing 260 00:17:28,350 --> 00:17:30,093 most commonly so far. 261 00:17:32,160 --> 00:17:37,160 Another very important part of social science research, 262 00:17:37,320 --> 00:17:41,640 especially when it's done as a profession as I do, 263 00:17:41,640 --> 00:17:44,190 is to use peer review. 264 00:17:44,190 --> 00:17:48,210 And we're gonna talk more about that in a few classes. 265 00:17:48,210 --> 00:17:51,714 But the idea being that I show my work 266 00:17:51,714 --> 00:17:55,470 to other experts in my field, my peers. 267 00:17:55,470 --> 00:17:58,140 They read my work, they think about it, 268 00:17:58,140 --> 00:18:01,710 they look at what I've done, was it well done? 269 00:18:01,710 --> 00:18:04,140 Are my methods sound? 270 00:18:04,140 --> 00:18:05,913 Are my results? 271 00:18:08,123 --> 00:18:10,406 Do they make sense? 272 00:18:10,406 --> 00:18:14,225 Are the conclusions I make on these results 273 00:18:14,225 --> 00:18:17,400 logical and sensible? 274 00:18:17,400 --> 00:18:20,097 And so by having outsiders, sort of, 275 00:18:20,097 --> 00:18:22,500 a set of fresh eyes, 276 00:18:22,500 --> 00:18:27,440 but who are also experts in the field, reviewing my work, 277 00:18:27,440 --> 00:18:32,440 hopefully that means that any work that I would report, 278 00:18:32,951 --> 00:18:36,600 once it has their sort of stamp of approval, 279 00:18:36,600 --> 00:18:41,193 will be much more credible as a result. 280 00:18:42,120 --> 00:18:46,075 And one of the most important things to address, 281 00:18:46,075 --> 00:18:51,075 especially social desirability bias or rather, sorry, 282 00:18:51,270 --> 00:18:54,240 that's another bias that we're gonna talk about. 283 00:18:54,240 --> 00:18:56,190 One of the most important things that we can do 284 00:18:56,190 --> 00:19:01,190 to deal with confirmation bias is reflexivity. 285 00:19:01,740 --> 00:19:03,840 And I'm gonna show you a slide about that. 286 00:19:09,600 --> 00:19:13,320 Here is a slide that talks about reflexivity 287 00:19:13,320 --> 00:19:15,450 or reflexive thinking. 288 00:19:15,450 --> 00:19:17,240 And I encourage you to read it 289 00:19:17,240 --> 00:19:21,090 and go to the cited link as well. 290 00:19:21,090 --> 00:19:24,030 The bottom line, for right now, 291 00:19:24,030 --> 00:19:27,330 is that it's really important for you 292 00:19:27,330 --> 00:19:31,714 to do your best to step outside of yourself 293 00:19:31,714 --> 00:19:35,070 and see yourself sort of from the outside, 294 00:19:35,070 --> 00:19:37,185 and think about what are the biases, 295 00:19:37,185 --> 00:19:40,950 what are the assumptions that I bring? 296 00:19:40,950 --> 00:19:45,950 Well, what am I more likely to think 297 00:19:46,320 --> 00:19:49,909 and see based on my own preconceptions? 298 00:19:49,909 --> 00:19:54,150 Because as this paragraph says, 299 00:19:54,150 --> 00:19:57,720 none of us are completely objective. 300 00:19:57,720 --> 00:20:02,720 And by very deliberately examining and owning our biases, 301 00:20:04,250 --> 00:20:09,250 it makes it much more likely that we will not fall prey 302 00:20:11,040 --> 00:20:14,010 as much to confirmation bias 303 00:20:14,010 --> 00:20:15,907 and will be more open-minded 304 00:20:15,907 --> 00:20:20,907 about evidence that may contradict 305 00:20:21,529 --> 00:20:24,543 our preexisting notions, 306 00:20:25,440 --> 00:20:29,841 and therefore, do a much better job at, you know, 307 00:20:29,841 --> 00:20:34,841 telling a more fair and full and credible account 308 00:20:35,070 --> 00:20:39,753 of the research issue that we are working on. 309 00:20:44,490 --> 00:20:49,490 Here are two cartoons, which I think spell things out well, 310 00:20:51,420 --> 00:20:54,390 especially the one, the top middle. 311 00:20:54,390 --> 00:20:58,170 Like, you see something, it's like click on it, 312 00:20:58,170 --> 00:21:02,103 look, it says what I thought it was, jackpot. 313 00:21:07,680 --> 00:21:10,830 So in the midst of our discussion 314 00:21:10,830 --> 00:21:14,310 when we are next together remotely, 315 00:21:14,310 --> 00:21:19,110 I want to explore a little bit more about confirmation bias. 316 00:21:19,110 --> 00:21:20,550 Why is it a concern? 317 00:21:20,550 --> 00:21:23,310 Why as consumers of research, 318 00:21:23,310 --> 00:21:27,540 why as producers of research is it a concern? 319 00:21:27,540 --> 00:21:32,190 Why is it something that we should really be a aware of 320 00:21:32,190 --> 00:21:33,840 and actively address? 321 00:21:33,840 --> 00:21:34,920 And how do we do that? 322 00:21:34,920 --> 00:21:36,453 How do we guard against it? 323 00:21:40,020 --> 00:21:41,610 The rest of this session, 324 00:21:41,610 --> 00:21:45,900 I would like to walk you through a number of concepts, 325 00:21:45,900 --> 00:21:49,800 vocabulary terms that we will be using 326 00:21:49,800 --> 00:21:52,833 throughout the rest of the class. 327 00:21:53,940 --> 00:21:55,751 Here's a list of them, 328 00:21:55,751 --> 00:21:59,343 and I will be going through each one of them. 329 00:22:05,550 --> 00:22:10,230 So the first two are two basic approaches 330 00:22:10,230 --> 00:22:15,230 of how do you collect and analyze the data? 331 00:22:16,230 --> 00:22:21,230 And I'll go through each one in the next few slides. 332 00:22:26,340 --> 00:22:28,770 The first is the distinction 333 00:22:28,770 --> 00:22:31,953 between idiographic and nomothetic. 334 00:22:32,940 --> 00:22:35,373 And this basically looks at, 335 00:22:40,290 --> 00:22:45,153 it's a function of whether you're looking at 336 00:22:45,153 --> 00:22:50,153 a few cases in great depth or looking 337 00:22:51,900 --> 00:22:56,900 at a narrower, shallower view of many cases. 338 00:23:00,060 --> 00:23:04,980 And hopefully, a few examples and more detail might help. 339 00:23:04,980 --> 00:23:06,930 I imagine both of these terms 340 00:23:06,930 --> 00:23:09,393 are fairly unfamiliar to you. 341 00:23:12,960 --> 00:23:15,992 When you take an idiographic approach, 342 00:23:15,992 --> 00:23:19,721 you are taking a very deep, detailed, 343 00:23:19,721 --> 00:23:24,721 exhaustive description of just a few events or a few cases. 344 00:23:28,260 --> 00:23:32,970 So you're sort of learning a lot about a little. 345 00:23:32,970 --> 00:23:33,849 So if your question 346 00:23:33,849 --> 00:23:37,640 is how do students maintain healthy weight? 347 00:23:37,640 --> 00:23:40,808 The idiographic approach might really focus 348 00:23:40,808 --> 00:23:43,080 on one per person. 349 00:23:43,080 --> 00:23:46,750 Like, this sentence here about my roommate 350 00:23:48,270 --> 00:23:51,480 and all the things that he does. 351 00:23:51,480 --> 00:23:53,820 A very sort of exhaustive, deep, 352 00:23:53,820 --> 00:23:58,820 detailed description of what this one individual does 353 00:23:59,160 --> 00:24:00,510 to maintain healthy weight. 354 00:24:04,410 --> 00:24:07,780 If you were to take a nomothetic approach, 355 00:24:07,780 --> 00:24:11,070 it would be looking at just a couple 356 00:24:11,070 --> 00:24:13,332 of explanatory variables 357 00:24:13,332 --> 00:24:16,110 that you think are of great importance 358 00:24:16,110 --> 00:24:18,540 and look at it at a number, 359 00:24:18,540 --> 00:24:22,746 a larger number of respondents or observation. 360 00:24:22,746 --> 00:24:26,670 And in this way, it's sort of learning a little about a lot, 361 00:24:26,670 --> 00:24:29,883 a little information about a lot of people. 362 00:24:30,780 --> 00:24:35,780 So here, this description here is a more nomothetic, 363 00:24:36,540 --> 00:24:39,570 where there was a survey of 200 students 364 00:24:39,570 --> 00:24:44,503 and found that these few factors here, 365 00:24:44,503 --> 00:24:48,210 fruit and vegetables, no alcohol, 366 00:24:48,210 --> 00:24:50,790 and getting adequate sleep, 367 00:24:50,790 --> 00:24:53,370 that these are the key factors 368 00:24:53,370 --> 00:24:56,340 that were found in maintaining healthy weight. 369 00:24:56,340 --> 00:24:59,400 So hopefully, this example makes sense 370 00:24:59,400 --> 00:25:04,400 of the idiographic is a lot of detailed information 371 00:25:04,950 --> 00:25:08,130 about a small number of individuals say, 372 00:25:08,130 --> 00:25:11,460 and the nomothetic is looking at 373 00:25:11,460 --> 00:25:13,260 just a little bit of information, 374 00:25:13,260 --> 00:25:18,230 a few key variables in a very large population. 375 00:25:21,060 --> 00:25:23,313 The other dialectic or number approach 376 00:25:23,313 --> 00:25:28,313 is more in the analysis of data 377 00:25:28,380 --> 00:25:30,360 and the direction of reasoning. 378 00:25:30,360 --> 00:25:34,470 And how do you sort of derive details 379 00:25:34,470 --> 00:25:36,963 from patterns and vice versa. 380 00:25:38,017 --> 00:25:40,694 So an inductive approach 381 00:25:40,694 --> 00:25:43,593 is where you start with the data, 382 00:25:44,760 --> 00:25:47,006 you start with the information, 383 00:25:47,006 --> 00:25:49,290 the observations that you do, 384 00:25:49,290 --> 00:25:52,433 and you look for patterns and themes 385 00:25:52,433 --> 00:25:56,670 and maybe develop hypotheses 386 00:25:56,670 --> 00:26:00,085 or general principles or theories 387 00:26:00,085 --> 00:26:04,380 that you can then test with later data. 388 00:26:04,380 --> 00:26:06,180 So this is sort of a bottom-up approach, 389 00:26:06,180 --> 00:26:10,470 where you start with the data themselves, the information, 390 00:26:10,470 --> 00:26:14,700 the observation themselves and look at it and say, 391 00:26:14,700 --> 00:26:16,290 what are the patterns emerging? 392 00:26:16,290 --> 00:26:19,410 Well, what are the themes that we're seeing in these data? 393 00:26:19,410 --> 00:26:22,737 And what sort of hypotheses or principles 394 00:26:24,018 --> 00:26:27,663 do these patterns and themes suggest? 395 00:26:28,950 --> 00:26:31,740 Whereas deductive, you're starting 396 00:26:31,740 --> 00:26:35,040 from a general principle 397 00:26:35,040 --> 00:26:37,892 that you are assuming is true and you say, 398 00:26:37,892 --> 00:26:39,360 well, if this is true, 399 00:26:39,360 --> 00:26:42,720 then this is true and this is true, then this is true. 400 00:26:42,720 --> 00:26:47,720 And you're starting with the principle 401 00:26:47,760 --> 00:26:51,330 and deriving observations from that. 402 00:26:51,330 --> 00:26:55,230 So this is a good way of testing hypotheses. 403 00:26:55,230 --> 00:26:57,570 And this tends to be a more top-down approach, 404 00:26:57,570 --> 00:27:01,290 where you start with the principle and see 405 00:27:01,290 --> 00:27:05,255 if the data hold true to the predictions 406 00:27:05,255 --> 00:27:10,255 or the forecasts of that sort of principle would suggest. 407 00:27:17,280 --> 00:27:22,280 Another set of terminology is these two types of analysis, 408 00:27:22,920 --> 00:27:25,590 positive and normative. 409 00:27:25,590 --> 00:27:30,583 So a positive analysis strictly describes, it says what is, 410 00:27:31,710 --> 00:27:34,590 not what should be, what's right or wrong, 411 00:27:34,590 --> 00:27:37,560 it's just what is present. 412 00:27:37,560 --> 00:27:42,150 And it tends to describe a phenomenon. 413 00:27:42,150 --> 00:27:46,471 Whereas a normative approach says what should be. 414 00:27:46,471 --> 00:27:50,100 And in this case, you would have to agree on 415 00:27:50,100 --> 00:27:52,290 what's the goal, what's the direction, 416 00:27:52,290 --> 00:27:54,870 what does better look like. 417 00:27:54,870 --> 00:27:57,570 And it tends to be more prescriptive. 418 00:27:57,570 --> 00:28:00,570 That it says this is what we should do 419 00:28:00,570 --> 00:28:03,870 to attain this goal, 420 00:28:03,870 --> 00:28:07,803 or this action is better or more preferred. 421 00:28:08,850 --> 00:28:12,830 So if I were to ask what teaching techniques do I use? 422 00:28:12,830 --> 00:28:16,500 That might lend itself to a positive approach. 423 00:28:16,500 --> 00:28:21,030 Whereas what teaching techniques should I use 424 00:28:21,030 --> 00:28:23,820 to create an effective space? 425 00:28:23,820 --> 00:28:25,710 Then that would be normative. 426 00:28:25,710 --> 00:28:30,633 The goal being creating effective learning for you all. 427 00:28:38,370 --> 00:28:43,370 So one of the practices that I have used 428 00:28:43,890 --> 00:28:47,280 throughout my career as a researcher 429 00:28:47,280 --> 00:28:52,089 and that I put a lot of faith and credit 430 00:28:52,089 --> 00:28:56,490 and find a lot of value in is using mixed methods. 431 00:28:56,490 --> 00:29:01,490 That thinking that most really complex 432 00:29:01,620 --> 00:29:06,620 and therefore interesting social science questions 433 00:29:08,160 --> 00:29:11,530 tend to have rather complex answers. 434 00:29:16,470 --> 00:29:20,010 And by using a mixed method, 435 00:29:20,010 --> 00:29:22,620 that they can create a more complete 436 00:29:22,620 --> 00:29:24,892 and comprehensive account 437 00:29:24,892 --> 00:29:27,810 of the thing that you are studying. 438 00:29:27,810 --> 00:29:32,810 So a qualitative approach would have non-numerical data. 439 00:29:32,910 --> 00:29:36,840 Usually, you're examining text of some kind, 440 00:29:36,840 --> 00:29:38,970 something that someone said or wrote, 441 00:29:38,970 --> 00:29:43,426 or behaviors that they are doing. 442 00:29:43,426 --> 00:29:47,010 And it tends to seek a much more in-depth understanding 443 00:29:47,010 --> 00:29:49,860 of social phenomena. 444 00:29:49,860 --> 00:29:52,380 It tends to be rather flexible. 445 00:29:52,380 --> 00:29:56,760 So you tend to ask a lot of open-ended questions, 446 00:29:56,760 --> 00:30:00,513 and tell me in your own words what is your experience? 447 00:30:01,770 --> 00:30:03,872 Whereas a quantitative, 448 00:30:03,872 --> 00:30:07,230 it tends to deal much more with numbers, 449 00:30:07,230 --> 00:30:10,980 where you have them choose from a number of options 450 00:30:10,980 --> 00:30:14,220 and then in, therefore, something like a survey, 451 00:30:14,220 --> 00:30:17,670 you can count how many said this versus how many said that, 452 00:30:17,670 --> 00:30:21,990 look at correlations of those who tended 453 00:30:21,990 --> 00:30:25,950 to answer true on this question 454 00:30:25,950 --> 00:30:29,220 were more or less likely to agree or not agree 455 00:30:29,220 --> 00:30:31,260 with this other question. 456 00:30:31,260 --> 00:30:33,390 Tends to be much more structured. 457 00:30:33,390 --> 00:30:36,770 And again, I'm a very firm believer 458 00:30:36,770 --> 00:30:39,870 in the value of mixed methods 459 00:30:39,870 --> 00:30:44,430 and believing that they really compliment each other well, 460 00:30:44,430 --> 00:30:48,030 they tell different sides of the story 461 00:30:48,030 --> 00:30:50,400 and together do a better job 462 00:30:50,400 --> 00:30:53,220 than either one singularly might do. 463 00:30:53,220 --> 00:30:55,324 And not so surprisingly, 464 00:30:55,324 --> 00:30:59,610 this class will deal with mixed methods throughout. 465 00:30:59,610 --> 00:31:04,080 We'll be employing a number of both qualitative 466 00:31:04,080 --> 00:31:07,020 and quantitative methods. 467 00:31:07,020 --> 00:31:08,280 We will learn about them, 468 00:31:08,280 --> 00:31:11,853 and we will be employing them in our research project. 469 00:31:16,830 --> 00:31:20,010 The next set of vocabulary is probably something 470 00:31:20,010 --> 00:31:22,500 you're a little bit more familiar with. 471 00:31:22,500 --> 00:31:24,993 That is variables and attributes. 472 00:31:26,160 --> 00:31:28,260 So variables, they vary. 473 00:31:28,260 --> 00:31:33,049 So, in theory, different people 474 00:31:33,049 --> 00:31:38,049 would give a different answer to each one. 475 00:31:39,240 --> 00:31:42,960 And you can think of it as questions on a survey. 476 00:31:42,960 --> 00:31:47,340 They tend to be measurable. 477 00:31:47,340 --> 00:31:52,340 And some demographic variables that you may be familiar with 478 00:31:52,350 --> 00:31:54,780 include age, occupation, income, 479 00:31:54,780 --> 00:31:58,840 race, gender, ethnicity, household size. 480 00:31:58,840 --> 00:32:03,180 So it's sort of like the questions on a survey. 481 00:32:03,180 --> 00:32:08,180 And if you asked everyone in this class these questions, 482 00:32:10,200 --> 00:32:13,500 the responses would vary among us, 483 00:32:13,500 --> 00:32:17,230 that we would have a variety of answers 484 00:32:18,300 --> 00:32:23,300 around age and race and gender and et cetera. 485 00:32:29,220 --> 00:32:34,220 By contrast, attributes are sort of the answer to a survey. 486 00:32:35,520 --> 00:32:40,520 It's each individual has an attribute for each variable. 487 00:32:42,120 --> 00:32:47,120 So your attribute for the variable of age, for many of you, 488 00:32:47,370 --> 00:32:52,370 may be 21 and your attribute for the variable 489 00:32:52,410 --> 00:32:54,960 of occupation would be student. 490 00:32:54,960 --> 00:32:58,590 Mine would obviously not be those. 491 00:32:58,590 --> 00:33:03,590 I am somewhat older than that, and I am a professor. 492 00:33:09,030 --> 00:33:11,310 I wanna say a bit about variable types, 493 00:33:11,310 --> 00:33:15,030 that sometimes we think about variables 494 00:33:15,030 --> 00:33:18,300 as being independent and dependent. 495 00:33:18,300 --> 00:33:22,110 So the independent variable 496 00:33:22,110 --> 00:33:26,460 is not the subject of the research. 497 00:33:26,460 --> 00:33:31,460 In some cases, it's not the behavior or action 498 00:33:31,988 --> 00:33:34,500 that we're really interested in, 499 00:33:34,500 --> 00:33:39,210 but we're wondering how a certain group 500 00:33:39,210 --> 00:33:42,780 of people might answer that question. 501 00:33:42,780 --> 00:33:44,924 So with the independent, 502 00:33:44,924 --> 00:33:49,924 we often think of it as given or exogenous 503 00:33:52,027 --> 00:33:56,940 and that in some cases, we might think of it as a cause. 504 00:33:56,940 --> 00:34:00,720 So in many cases, demographic variables 505 00:34:00,720 --> 00:34:02,850 may be the independent. 506 00:34:02,850 --> 00:34:06,540 One's age, income, household size, occupation, 507 00:34:06,540 --> 00:34:10,230 race, gender, gender identity, things like that. 508 00:34:10,230 --> 00:34:15,090 The dependent is the subject of the research. 509 00:34:15,090 --> 00:34:18,690 So in many cases, it is the behavior 510 00:34:18,690 --> 00:34:23,250 or the action or the attitude that we are interested in. 511 00:34:23,250 --> 00:34:26,914 And in some cases, we look at the relationship, 512 00:34:26,914 --> 00:34:31,914 there may even be a causal one by the independent. 513 00:34:32,490 --> 00:34:37,490 So again, in many cases, but not all, 514 00:34:37,860 --> 00:34:41,010 the demographic is an independent 515 00:34:41,010 --> 00:34:44,070 and the behavior is the dependent. 516 00:34:44,070 --> 00:34:47,098 So we might graph out how does, 517 00:34:47,098 --> 00:34:50,160 if we took a whole bunch of folks 518 00:34:50,160 --> 00:34:53,250 and made them run 100 meter dash, 519 00:34:53,250 --> 00:34:56,880 and we would graph age and speed 520 00:34:56,880 --> 00:35:00,660 that we would probably see a relationship 521 00:35:00,660 --> 00:35:05,660 where the speed is probably fastest or the lowest time 522 00:35:06,327 --> 00:35:10,500 somewhere in your teens and 20s. 523 00:35:10,500 --> 00:35:12,960 And then as you get older, 524 00:35:12,960 --> 00:35:15,330 the age would probably increase over time 525 00:35:15,330 --> 00:35:18,090 for the majority of folks. 526 00:35:18,090 --> 00:35:20,490 So you can maybe think of another example 527 00:35:20,490 --> 00:35:24,663 of what would be an independent and a dependent. 528 00:35:29,220 --> 00:35:34,220 The last topic is thinking about the quality of measurement 529 00:35:34,560 --> 00:35:39,560 and sort of what are the criteria for the quality 530 00:35:40,758 --> 00:35:43,170 of a measurement that we do? 531 00:35:43,170 --> 00:35:45,457 And I'm gonna talk about precision, 532 00:35:45,457 --> 00:35:50,100 accuracy, reliability, and validity. 533 00:35:50,100 --> 00:35:51,843 And I'm gonna talk about each one. 534 00:35:54,900 --> 00:35:58,920 So first I'm gonna talk about precision and accuracy. 535 00:35:58,920 --> 00:36:03,920 So precision is about the narrowness of detail, 536 00:36:04,072 --> 00:36:09,072 that how sort of narrow does the description go? 537 00:36:09,210 --> 00:36:12,450 So you see a photo of this man here, 538 00:36:12,450 --> 00:36:15,780 probably most of you know that is Tom Brady. 539 00:36:15,780 --> 00:36:19,157 And each of these attributes is true. 540 00:36:24,630 --> 00:36:26,700 So they are all accurate. 541 00:36:26,700 --> 00:36:29,610 So it is true that he is a carbon-based life form, 542 00:36:29,610 --> 00:36:31,260 he's a human, he's a male human, 543 00:36:31,260 --> 00:36:32,610 he plays in the NFL, 544 00:36:32,610 --> 00:36:34,740 plays for the Bucs, plays quarterback. 545 00:36:34,740 --> 00:36:39,740 And you can also see that these are sort of nested, 546 00:36:40,140 --> 00:36:43,560 that if you play quarterback for the Bucs, 547 00:36:43,560 --> 00:36:45,180 then you play for the Bucs. 548 00:36:45,180 --> 00:36:48,240 If you play for the Bucs, you play for the NFL. 549 00:36:48,240 --> 00:36:50,700 If you're in the NFL, you're a male human. 550 00:36:50,700 --> 00:36:52,320 All male humans are humans, 551 00:36:52,320 --> 00:36:54,360 and all humans are carbon-based life forms. 552 00:36:54,360 --> 00:36:57,660 You can see that we're coming from a very, 553 00:36:57,660 --> 00:37:01,710 very broad description to a very, very narrow one, 554 00:37:01,710 --> 00:37:02,730 down to, you know, 555 00:37:02,730 --> 00:37:07,315 one or a very small handful of individuals. 556 00:37:07,315 --> 00:37:12,315 To say Tom Brady plays quarterback for the Buccaneers 557 00:37:14,550 --> 00:37:16,740 is a much more precise statement 558 00:37:16,740 --> 00:37:20,280 than Tom Brady is a carbon-based life form. 559 00:37:20,280 --> 00:37:22,743 Even though they're both accurate. 560 00:37:23,580 --> 00:37:26,460 And accurate basically means it's true, 561 00:37:26,460 --> 00:37:30,630 it correctly reflects the the real world. 562 00:37:30,630 --> 00:37:33,090 And again, all of these statements are accurate, 563 00:37:33,090 --> 00:37:35,850 but each one, as we go from top to bottom, 564 00:37:35,850 --> 00:37:37,233 gets more precise. 565 00:37:40,110 --> 00:37:43,263 So do you think that it's important? 566 00:37:44,100 --> 00:37:47,370 Is one more important than the other? 567 00:37:47,370 --> 00:37:49,924 Is it more important to be accurate? 568 00:37:49,924 --> 00:37:53,177 Is it more important to be precise? 569 00:37:53,177 --> 00:37:54,483 And why? 570 00:37:55,380 --> 00:37:58,502 Or maybe think of it as I frame it there, true or false, 571 00:37:58,502 --> 00:38:03,303 accuracy is more important than precision. 572 00:38:06,870 --> 00:38:09,870 Well, I would say that it is a false statement, 573 00:38:09,870 --> 00:38:11,220 and here's why. 574 00:38:11,220 --> 00:38:15,330 That if you are going to locate me, 575 00:38:15,330 --> 00:38:19,860 so right now I am at my home most likely 576 00:38:19,860 --> 00:38:22,290 in Burlington, Vermont. 577 00:38:22,290 --> 00:38:26,283 So if you were to find me, and on my street, 578 00:38:29,456 --> 00:38:34,456 you would maybe say, you know, 579 00:38:35,827 --> 00:38:39,603 it would be an accurate statement 580 00:38:39,603 --> 00:38:44,603 to say Professor Connor is in the known universe, 581 00:38:48,450 --> 00:38:53,250 or in the solar system, or he's on Earth, 582 00:38:53,250 --> 00:38:58,250 or he is in North America, or he is in Vermont. 583 00:39:01,650 --> 00:39:05,310 And those are all accurate, but they're not very precise. 584 00:39:05,310 --> 00:39:07,713 But if you wanted to find me, you know, 585 00:39:09,180 --> 00:39:11,230 that there was some important reason, 586 00:39:11,230 --> 00:39:14,070 and they were going house to house, 587 00:39:14,070 --> 00:39:17,970 that you could say that I was one house over 588 00:39:17,970 --> 00:39:21,090 than my actual house. 589 00:39:21,090 --> 00:39:25,680 So 1 Main Street instead of 3 Main Street. 590 00:39:25,680 --> 00:39:28,863 And then that even though, 591 00:39:30,750 --> 00:39:32,943 if I'm actually at 1 Main street, 592 00:39:34,920 --> 00:39:39,480 but you say I'm at 3, that it would be a lot better, 593 00:39:39,480 --> 00:39:41,935 it could be a lot easier and quicker 594 00:39:41,935 --> 00:39:46,110 to find me by saying I'm on 3 Main Street, 595 00:39:46,110 --> 00:39:49,350 which is precise and not accurate 596 00:39:49,350 --> 00:39:53,760 than to say I am in North America, 597 00:39:53,760 --> 00:39:57,840 which is accurate but not precise. 598 00:39:57,840 --> 00:39:59,220 In the same way, 599 00:39:59,220 --> 00:40:02,250 if you're baking a cake and you need exactly 600 00:40:02,250 --> 00:40:04,620 one cup of milk, 601 00:40:04,620 --> 00:40:09,620 that an accurate response might be you need between 1/8 602 00:40:15,851 --> 00:40:18,720 and eight cups of milk. 603 00:40:18,720 --> 00:40:21,900 Okay, it's a very accurate statement, but it's not precise. 604 00:40:21,900 --> 00:40:26,550 But if you said 1.00001 cups of milk, 605 00:40:26,550 --> 00:40:29,310 it would not be completely accurate, 606 00:40:29,310 --> 00:40:32,130 but it would be much more precise, 607 00:40:32,130 --> 00:40:34,865 and that would be a much more useful bit of information 608 00:40:34,865 --> 00:40:37,143 to make a good cake. 609 00:40:41,310 --> 00:40:43,923 Finally, I'm going to talk about reliability. 610 00:40:45,090 --> 00:40:50,090 So reliability means that if you do the same procedure 611 00:40:50,670 --> 00:40:51,600 over and over, 612 00:40:51,600 --> 00:40:55,170 that you're going to get the same kinds of information. 613 00:40:55,170 --> 00:40:59,640 So another way of thinking about it is it's predictable. 614 00:40:59,640 --> 00:41:04,577 So a very good example is if you ask someone 615 00:41:06,540 --> 00:41:10,560 to answer a survey, pick one of these five things. 616 00:41:10,560 --> 00:41:13,047 Then you know that they're gonna pick 617 00:41:13,047 --> 00:41:15,030 one of these five things. 618 00:41:15,030 --> 00:41:20,030 Like Professor Connor is an effective professor 619 00:41:22,530 --> 00:41:24,270 and you can do a five point scale, 620 00:41:24,270 --> 00:41:28,170 strongly disagree to strongly agree, 621 00:41:28,170 --> 00:41:32,310 and you know that if they answer it and they do it right, 622 00:41:32,310 --> 00:41:34,440 they're gonna get one of those five things. 623 00:41:34,440 --> 00:41:36,141 Whereas if you said, 624 00:41:36,141 --> 00:41:41,141 what do you think about Professor Connor? 625 00:41:41,520 --> 00:41:42,870 It could be all over the place, 626 00:41:42,870 --> 00:41:46,110 that you could talk about me at at my job. 627 00:41:46,110 --> 00:41:47,550 You could talk about, I don't know, 628 00:41:47,550 --> 00:41:51,210 how I dress or what type of car I drive, 629 00:41:51,210 --> 00:41:56,210 or how funny my jokes are, or any of a number of things. 630 00:41:56,430 --> 00:41:59,412 So reliability means that you're gonna get 631 00:41:59,412 --> 00:42:01,650 the same kind of answers, 632 00:42:01,650 --> 00:42:06,650 and you tend to get a more precise measure from reliability. 633 00:42:09,900 --> 00:42:13,110 Validity means, basically, is it truthful? 634 00:42:13,110 --> 00:42:15,476 Does it credibly measure what you want to? 635 00:42:15,476 --> 00:42:18,780 Does it accomplish the purpose? 636 00:42:18,780 --> 00:42:22,650 So in other words, is it accurate? 637 00:42:22,650 --> 00:42:26,523 So valid means does it really reflect the real world? 638 00:42:28,710 --> 00:42:33,710 So if you're conducting an interview, 639 00:42:35,400 --> 00:42:38,400 instead of saying pick one of these five things, say, 640 00:42:38,400 --> 00:42:42,450 put your experience of this phenomenon 641 00:42:42,450 --> 00:42:45,870 that we're studying into your own words, 642 00:42:45,870 --> 00:42:50,520 and it will much more accurately 643 00:42:50,520 --> 00:42:54,330 and validly reflect what they actually think, 644 00:42:54,330 --> 00:42:55,756 that when because maybe 645 00:42:55,756 --> 00:42:59,940 none of the five things that you ask on a survey 646 00:42:59,940 --> 00:43:04,830 are really right or maybe the survey question that you ask, 647 00:43:04,830 --> 00:43:07,830 they really don't even know or don't have an opinion 648 00:43:07,830 --> 00:43:09,753 or don't find it important. 649 00:43:11,820 --> 00:43:13,830 We're gonna spend a lot of time 650 00:43:13,830 --> 00:43:18,150 with accuracy and reliability over time. 651 00:43:18,150 --> 00:43:20,823 But I wanted to introduce it now. 652 00:43:26,430 --> 00:43:28,260 This slide sort of sums it up. 653 00:43:28,260 --> 00:43:32,460 So when you do nomothetic research, 654 00:43:32,460 --> 00:43:37,460 remember that that is asking just a few variables 655 00:43:38,850 --> 00:43:42,180 and getting the attributes of a large number of people. 656 00:43:42,180 --> 00:43:43,755 It tends to be much more structured, 657 00:43:43,755 --> 00:43:45,480 tend to be quantitative 658 00:43:45,480 --> 00:43:47,970 because we count how many say each thing, 659 00:43:47,970 --> 00:43:50,100 tend to be very reliable 660 00:43:50,100 --> 00:43:54,390 because you only get a narrow range of answers, 661 00:43:54,390 --> 00:43:57,060 tends to be quite predictable. 662 00:43:57,060 --> 00:44:01,920 The downside is it can be rather shallow or superficial 663 00:44:01,920 --> 00:44:02,753 when you're saying 664 00:44:02,753 --> 00:44:05,141 choose one of these five things, and maybe, 665 00:44:05,141 --> 00:44:09,240 you know, again, maybe you're asking the wrong question, 666 00:44:09,240 --> 00:44:11,921 maybe none of those five things apply. 667 00:44:11,921 --> 00:44:16,530 Whereas when you do an idiographic approach, 668 00:44:16,530 --> 00:44:18,120 it's much more flexible. 669 00:44:18,120 --> 00:44:19,650 You get qualitative data 670 00:44:19,650 --> 00:44:21,528 'cause they express it in their own words. 671 00:44:21,528 --> 00:44:23,370 Tends to be very valid. 672 00:44:23,370 --> 00:44:26,371 But it also then tends to be very unpredictable, 673 00:44:26,371 --> 00:44:28,604 that it's, in many ways, 674 00:44:28,604 --> 00:44:33,604 a lot easier to analyze quantitative data 675 00:44:34,198 --> 00:44:36,360 than it is qualitative data 676 00:44:36,360 --> 00:44:38,377 and much more sort of straightforward 677 00:44:38,377 --> 00:44:41,223 because of its reliability. 678 00:44:45,480 --> 00:44:50,480 So both of these approaches are commonly used 679 00:44:50,910 --> 00:44:53,910 in social and economic research, 680 00:44:53,910 --> 00:44:55,037 that in many cases, 681 00:44:55,037 --> 00:44:58,350 when you do an idiographic approach, 682 00:44:58,350 --> 00:45:00,780 you use things like focus groups 683 00:45:00,780 --> 00:45:03,810 and key informant interviews where you talk to people, 684 00:45:03,810 --> 00:45:08,160 you have 'em put their experiences into their own words. 685 00:45:08,160 --> 00:45:09,930 You probably only talk 686 00:45:09,930 --> 00:45:13,050 to a fairly small number of respondents, 687 00:45:13,050 --> 00:45:15,240 whereas as a nomothetic, 688 00:45:15,240 --> 00:45:17,051 you would administer a survey, 689 00:45:17,051 --> 00:45:20,880 and you get lots and lots and lots of responses. 690 00:45:20,880 --> 00:45:23,880 You know, you can get data from lots of people, 691 00:45:23,880 --> 00:45:27,873 but there's only so many sort of questions that you can ask. 692 00:45:30,480 --> 00:45:35,480 Here are some key takeaways, 693 00:45:35,520 --> 00:45:39,390 and I encourage you to look these over 694 00:45:39,390 --> 00:45:41,939 and make sure that they all make sense 695 00:45:41,939 --> 00:45:46,939 and be ready to discuss them 696 00:45:47,400 --> 00:45:52,400 when we come to together in the remote class setting.