WEBVTT 1 00:00:01.230 --> 00:00:05.580 Hello and welcome to part two of "Qualitative Research." 2 00:00:05.580 --> 00:00:07.830 So in this part we're gonna be talking much more about 3 00:00:07.830 --> 00:00:10.200 sort of the nuts and bolts of the logistics 4 00:00:10.200 --> 00:00:12.060 of how do you do it, 5 00:00:12.060 --> 00:00:14.910 especially how do you collect the data 6 00:00:14.910 --> 00:00:17.250 and how do you sort of prepare yourself 7 00:00:17.250 --> 00:00:22.250 so you get really good quality data from your efforts. 8 00:00:24.720 --> 00:00:28.830 So we're going to talk about three main things here. 9 00:00:28.830 --> 00:00:31.650 How do you structure an interview, 10 00:00:31.650 --> 00:00:34.833 how much structure as we'll see, 11 00:00:35.700 --> 00:00:37.320 types of questions 12 00:00:37.320 --> 00:00:39.690 and what is a good order of questions. 13 00:00:39.690 --> 00:00:43.350 So as you sort of have questions that you wanna ask 14 00:00:43.350 --> 00:00:45.637 and you're developing interview guide. 15 00:00:45.637 --> 00:00:48.810 What are the guidelines for the order 16 00:00:48.810 --> 00:00:52.410 of questions that you will ask, 17 00:00:52.410 --> 00:00:54.933 what do you ask first, what do you ask last. 18 00:00:56.910 --> 00:00:59.760 So there's three main types of structure. 19 00:00:59.760 --> 00:01:02.130 First there is structured 20 00:01:02.130 --> 00:01:05.130 and that is where you have a very precise script. 21 00:01:05.130 --> 00:01:06.780 You read it the same way. 22 00:01:06.780 --> 00:01:11.130 You always ask the same questions in the same order 23 00:01:11.130 --> 00:01:13.110 for all of your subjects. 24 00:01:13.110 --> 00:01:15.630 You don't sort of deviate from that. 25 00:01:15.630 --> 00:01:19.980 And in that way you tend to get more reliable answers. 26 00:01:19.980 --> 00:01:21.450 They're very predictable 27 00:01:21.450 --> 00:01:25.920 that you're asking specific questions in a specific order. 28 00:01:25.920 --> 00:01:30.003 And that sort of adds some reliability to the process. 29 00:01:32.220 --> 00:01:34.890 At the other end is an unstructured, 30 00:01:34.890 --> 00:01:36.300 where it's much more freeform. 31 00:01:36.300 --> 00:01:40.140 You may use it to just sort of have key points 32 00:01:40.140 --> 00:01:43.200 that you wish to cover, 33 00:01:43.200 --> 00:01:47.910 but you are free to drop or add questions, 34 00:01:47.910 --> 00:01:50.520 sort of deviate from the script. 35 00:01:50.520 --> 00:01:52.410 And in this way 36 00:01:52.410 --> 00:01:57.180 because you're sort of letting one response lead 37 00:01:57.180 --> 00:01:59.670 to the next question 38 00:01:59.670 --> 00:02:02.730 and maybe you're getting much more valid questions. 39 00:02:02.730 --> 00:02:07.730 And again, because you're letting the interviewee 40 00:02:07.830 --> 00:02:12.000 sort of guide the process a lot more. 41 00:02:12.000 --> 00:02:16.560 And as always, you're letting them express themselves 42 00:02:16.560 --> 00:02:17.880 in their own words, 43 00:02:17.880 --> 00:02:21.393 which is a good way to get valid answers. 44 00:02:23.790 --> 00:02:27.720 Why is it that if you have multiple interviewers 45 00:02:27.720 --> 00:02:32.250 like we'll be doing in this project, 46 00:02:32.250 --> 00:02:34.740 why do you want a structured one? 47 00:02:34.740 --> 00:02:37.623 And we'll discuss that in class. 48 00:02:39.420 --> 00:02:43.320 So the in-between case is probably the one that's used most. 49 00:02:43.320 --> 00:02:45.960 This is so-called semi-structured interviews. 50 00:02:45.960 --> 00:02:50.280 And this sort of is in a sense 51 00:02:50.280 --> 00:02:51.720 sort of the best of both worlds, 52 00:02:51.720 --> 00:02:54.300 takes the strength of each one, 53 00:02:54.300 --> 00:02:56.850 that there's some structure which makes sure 54 00:02:56.850 --> 00:02:58.890 that you hit the key points, 55 00:02:58.890 --> 00:03:02.970 that you make sure that you get data on the questions 56 00:03:02.970 --> 00:03:05.340 that you wanna know about most, 57 00:03:05.340 --> 00:03:08.010 but it also allows for some flexibility. 58 00:03:08.010 --> 00:03:11.610 And as you learn more, you can sort of tweak, 59 00:03:11.610 --> 00:03:13.230 or add new questions, 60 00:03:13.230 --> 00:03:16.830 or drop ones that you find really aren't 61 00:03:16.830 --> 00:03:21.150 yielding anything of interest or value. 62 00:03:21.150 --> 00:03:25.350 And this is probably the most common structure 63 00:03:25.350 --> 00:03:28.863 that's used in interviews by most social scientists. 64 00:03:31.350 --> 00:03:34.110 So when you're doing interviews, 65 00:03:34.110 --> 00:03:37.410 you wanna set an easy conversational tone. 66 00:03:37.410 --> 00:03:41.200 You want your subject to be relaxed and open, 67 00:03:45.450 --> 00:03:47.490 not on guard. 68 00:03:47.490 --> 00:03:50.220 You sort of let, especially with semi-structured, 69 00:03:50.220 --> 00:03:54.870 you let the responses shape the next question. 70 00:03:54.870 --> 00:03:59.580 And it's always a good idea to have some probes in mind 71 00:03:59.580 --> 00:04:02.160 where you say, "Tell me more about this," 72 00:04:02.160 --> 00:04:04.170 or, "Why does it work like that?" 73 00:04:04.170 --> 00:04:06.990 or, "What's an example of this?" 74 00:04:06.990 --> 00:04:09.090 And as with so many other things, 75 00:04:09.090 --> 00:04:13.290 the best way to get really good at being an interviewer 76 00:04:13.290 --> 00:04:15.600 is to do lots of practice. 77 00:04:15.600 --> 00:04:18.333 The more you do it, the better you become. 78 00:04:20.730 --> 00:04:23.760 So next, I wanna talk a bit about 79 00:04:23.760 --> 00:04:27.180 open versus closed ended question. 80 00:04:27.180 --> 00:04:31.410 So closed ended is you have them choose 81 00:04:31.410 --> 00:04:32.760 from one of these things. 82 00:04:32.760 --> 00:04:36.060 Choose one of these five things reverses open-ended 83 00:04:36.060 --> 00:04:41.060 is that you let them phrase the answer in their own words 84 00:04:41.070 --> 00:04:45.360 and think about which one will yield more reliable results 85 00:04:45.360 --> 00:04:49.023 and which one will yield more valid results. 86 00:04:50.430 --> 00:04:51.930 So think about that 87 00:04:51.930 --> 00:04:56.733 and the next slide will say which is which. 88 00:04:58.170 --> 00:05:00.990 So hopefully this makes sense 89 00:05:00.990 --> 00:05:04.230 that if you're saying choose from one of these things, 90 00:05:04.230 --> 00:05:07.680 you know they're gonna choose one of those five things. 91 00:05:07.680 --> 00:05:10.260 Whereas when you're open-ended, 92 00:05:10.260 --> 00:05:12.180 you're letting them express it in their own words 93 00:05:12.180 --> 00:05:13.930 and it tends to be much more valid. 94 00:05:15.270 --> 00:05:17.430 All right, so when you're thinking about 95 00:05:17.430 --> 00:05:20.400 how do you structure an interview guide, 96 00:05:20.400 --> 00:05:21.900 these are three guidelines. 97 00:05:21.900 --> 00:05:25.500 Note that they are not mutually exclusive. 98 00:05:25.500 --> 00:05:28.980 There are other considerations, 99 00:05:28.980 --> 00:05:33.483 they may even come into conflict. 100 00:05:34.380 --> 00:05:37.290 But here are sort of three guidelines 101 00:05:37.290 --> 00:05:39.900 that I think it's important to follow. 102 00:05:39.900 --> 00:05:44.250 First is you ask more general questions first 103 00:05:44.250 --> 00:05:48.420 and then more specific questions second. 104 00:05:48.420 --> 00:05:53.420 So an example of this is if you're interviewing students 105 00:05:54.450 --> 00:05:57.480 and you wanna know about sort of their 106 00:05:57.480 --> 00:06:01.770 general experience in the classroom 107 00:06:01.770 --> 00:06:06.030 as opposed to their specific experience with one class, 108 00:06:06.030 --> 00:06:08.493 you would ask the general one first. 109 00:06:09.510 --> 00:06:13.747 And you do that because if you ask say, 110 00:06:13.747 --> 00:06:18.327 "What were your experiences in CDAE 3500?" 111 00:06:19.267 --> 00:06:22.590 They very well might have that on their mind 112 00:06:22.590 --> 00:06:27.247 when you go to, "Well, tell me about your experiences 113 00:06:27.247 --> 00:06:30.780 "as a UVM student?" 114 00:06:30.780 --> 00:06:34.470 So you're already sort of flavoring their questions 115 00:06:34.470 --> 00:06:37.530 when you ask the specific ones. 116 00:06:37.530 --> 00:06:39.870 And again, you wanna move down that funnel 117 00:06:39.870 --> 00:06:42.630 from general to specific. 118 00:06:42.630 --> 00:06:44.620 Most to least important 119 00:06:45.720 --> 00:06:50.070 because they might, you know, you might run out of time, 120 00:06:50.070 --> 00:06:52.620 they may not want to do it anymore, 121 00:06:52.620 --> 00:06:55.350 get to the more important information first 122 00:06:55.350 --> 00:06:58.710 and last, safest to riskiest, 123 00:06:58.710 --> 00:07:03.030 that if you're going to ask questions that have high risk, 124 00:07:03.030 --> 00:07:06.450 that they may be offended by maybe, you know, 125 00:07:06.450 --> 00:07:09.780 sort of like things that might put them, 126 00:07:09.780 --> 00:07:14.370 make them vulnerable like sexuality, 127 00:07:14.370 --> 00:07:17.610 or like, you know, drug use or things like that. 128 00:07:17.610 --> 00:07:20.423 Or just, you know, sort of political views 129 00:07:24.780 --> 00:07:29.520 and more sort of, it is sadly in my mind 130 00:07:29.520 --> 00:07:33.060 becoming less safe to express political views. 131 00:07:33.060 --> 00:07:35.460 So you wanna do the safest one 132 00:07:35.460 --> 00:07:37.170 because it might be, 133 00:07:37.170 --> 00:07:40.680 if you ask a really sensitive, risky question, 134 00:07:40.680 --> 00:07:42.847 they might just sort of stop and say, 135 00:07:42.847 --> 00:07:44.760 "I don't want to do this." 136 00:07:44.760 --> 00:07:49.760 And the Human Subjects protocols say that 137 00:07:52.410 --> 00:07:54.630 when someone doesn't, you know, 138 00:07:54.630 --> 00:07:57.030 they don't have to answer a question, 139 00:07:57.030 --> 00:07:59.670 they can quit at any time. 140 00:07:59.670 --> 00:08:03.450 So you want to make sure that you get, you know, 141 00:08:03.450 --> 00:08:06.870 save those really risky ones for the end 142 00:08:06.870 --> 00:08:11.013 in case that's the very last question that you get to ask. 143 00:08:14.760 --> 00:08:19.440 The most important ingredient in a good interview is you. 144 00:08:19.440 --> 00:08:23.060 A thinking, observing, present researcher. 145 00:08:26.070 --> 00:08:28.890 You want to at very least take notes 146 00:08:28.890 --> 00:08:33.890 in a lot of sort of, you know, thesis type of work. 147 00:08:35.940 --> 00:08:38.910 You record it, 148 00:08:38.910 --> 00:08:41.070 but it's important to both take notes 149 00:08:41.070 --> 00:08:44.490 and then go back and revise and reorganize 150 00:08:44.490 --> 00:08:47.190 and see what's there, take a lot of notes, 151 00:08:47.190 --> 00:08:50.913 and again, the best way to get good at it is to practice. 152 00:08:53.400 --> 00:08:57.870 So here are some of the strengths and weaknesses overall 153 00:08:57.870 --> 00:08:59.610 of qualitative. 154 00:08:59.610 --> 00:09:01.473 They tend to be very flexible. 155 00:09:02.310 --> 00:09:05.880 You get a great depth and they are very valid. 156 00:09:05.880 --> 00:09:09.360 The weakness is that in most cases, 157 00:09:09.360 --> 00:09:14.160 no sort of statistical analysis or inference. 158 00:09:14.160 --> 00:09:15.540 You get a lack of breadth 159 00:09:15.540 --> 00:09:19.050 that you don't really have a lot of time. 160 00:09:19.050 --> 00:09:21.510 You know, since each interview takes a long time 161 00:09:21.510 --> 00:09:23.460 and yields a lot of data, 162 00:09:23.460 --> 00:09:25.710 you tend to speak with fewer of them. 163 00:09:25.710 --> 00:09:28.680 And again, it's not as reliable 164 00:09:28.680 --> 00:09:31.590 because you're letting them 165 00:09:31.590 --> 00:09:33.390 express things in their own words 166 00:09:33.390 --> 00:09:36.000 and they can sort of take things in the direction 167 00:09:36.000 --> 00:09:39.750 that they want instead of choose 168 00:09:39.750 --> 00:09:42.693 which of these five things is most true for you. 169 00:09:45.270 --> 00:09:49.533 So again, a strength is that it's valid, it's firsthand, 170 00:09:50.785 --> 00:09:54.510 direct accounts that they will probably in fact 171 00:09:54.510 --> 00:09:55.830 that they should bring up things 172 00:09:55.830 --> 00:09:57.390 that you wouldn't have thought of. 173 00:09:57.390 --> 00:10:00.240 That, you know, if you just started with a survey, 174 00:10:00.240 --> 00:10:01.620 there's probably a lot of things 175 00:10:01.620 --> 00:10:04.440 that you just don't know enough to ask. 176 00:10:04.440 --> 00:10:08.190 So doing interviews first can help make sure 177 00:10:08.190 --> 00:10:13.190 that when you do do a survey that you ask the right things 178 00:10:13.560 --> 00:10:16.830 and note that as well that a qualitative piece 179 00:10:16.830 --> 00:10:19.200 can stand by itself too. 180 00:10:19.200 --> 00:10:22.830 It's not just a means to the end of a survey, 181 00:10:22.830 --> 00:10:25.473 but it also has its own value. 182 00:10:28.890 --> 00:10:33.633 And again, it tends to be qualitative can be less reliable, 183 00:10:35.040 --> 00:10:38.310 you know, it's a very sort of personal, 184 00:10:38.310 --> 00:10:40.320 it's their own views. 185 00:10:40.320 --> 00:10:44.280 So we need to be aware of our own biases, as I said, 186 00:10:44.280 --> 00:10:48.720 and use comparisons and be specific. 187 00:10:48.720 --> 00:10:52.860 And here's sort of a phrase to a good way 188 00:10:52.860 --> 00:10:57.660 of the difference between a senior 189 00:10:57.660 --> 00:11:00.090 and a sophomore's responses. 190 00:11:00.090 --> 00:11:03.150 The top one, you know, very specific, 191 00:11:03.150 --> 00:11:08.150 whereas the second one is much more vague. 192 00:11:08.220 --> 00:11:10.920 And this, you know, I think one of the weaknesses 193 00:11:10.920 --> 00:11:14.250 or the perceived weaknesses of qualitative is, 194 00:11:14.250 --> 00:11:16.620 well that's just your view. 195 00:11:16.620 --> 00:11:19.920 I just should take your word for it. 196 00:11:19.920 --> 00:11:22.050 And by being specific like this, 197 00:11:22.050 --> 00:11:24.090 it sort of adds meat onto those bones 198 00:11:24.090 --> 00:11:27.873 and it makes the results much more credible. 199 00:11:29.760 --> 00:11:32.580 So here are some things to look at 200 00:11:32.580 --> 00:11:35.940 for a qualitative piece if it's done well. 201 00:11:35.940 --> 00:11:38.883 So first did they do a good lit review? 202 00:11:40.621 --> 00:11:41.753 Does the sample make sense? 203 00:11:43.200 --> 00:11:45.250 Do the people that they talked to 204 00:11:46.500 --> 00:11:49.020 make sense for what they're doing? 205 00:11:49.020 --> 00:11:53.070 Did they sort of seek out deviant cases 206 00:11:56.310 --> 00:12:00.960 and not just talk to people that all hold the same views, 207 00:12:00.960 --> 00:12:05.960 and we're gonna learn about of coding and reporting. 208 00:12:07.170 --> 00:12:11.910 But when they do the analysis, are they transparent? 209 00:12:11.910 --> 00:12:15.900 It's very good when you analyze the data 210 00:12:15.900 --> 00:12:20.900 to use multiple coders, which adds to the reliability. 211 00:12:22.920 --> 00:12:27.920 And last when they report out, is that also transparent? 212 00:12:28.920 --> 00:12:31.380 And can you sort of hear and understand 213 00:12:31.380 --> 00:12:33.900 the voice of the respondents 214 00:12:33.900 --> 00:12:38.900 and not just the interpretation of the researcher. 215 00:12:40.500 --> 00:12:42.273 So this is what we did. 216 00:12:44.460 --> 00:12:45.363 And thank you.