[Instructor] Hello everyone and welcome to this session
where we're gonna be talking about
how to do a literature review.
How do you read through, collect and organize,
and write up the results of a review of what research
has been done on the topic that you are studying?
This will be your first homework assignment
and this is helping you to get ready for that.
So, let's start to talk about what is a literature review.
So, it's a summary of previous research on a topic.
So, you collect, organize, analyze
and report out the findings of previous research
on the topic that you are studying.
And again, its purpose is to review
what do we already know, to lay out a case
of this is what we already know.
And then, with the intent of building on
what has already been done.
The two key questions that a lit review answers are,
what is known about the subject?
What do we already know, and what are the key gaps?
What do we not know?
And hopefully, your work will be to fill in
one or more of those gaps.
Some of the questions that you'll ask
as you go through, are what are the methods
or the problems that have identified in previous research
and how do they impact your research?
And to sort of bring the reader up to date
of what is the current status of research in this area?
Here are my thoughts on why you should do a lit review.
First, it's to help you to design your study,
so you're not reinventing the wheel.
So, you are covering fresh ground as well as
by situating it in what's already been done,
it helps the readers to interpret the results.
It also explains how your research question
fits into the larger picture of what we already know.
It brings the reader up to date, it justifies your approach.
So, if you do a good job of outlining what we already know
and what we don't know, that serves as a good justification
of the work that you are proposing to do.
And last, it makes you look more credible,
that by showing that you understand
what has already been done, it establishes
that you've done your homework,
you've done your due diligence, and that it makes
what you say more credible.
How do you do it?
So, you basically summarize and explain what's been done,
always citing the sources as you mention them.
You can point out the different way that researchers
had treated the topic.
And again, explaining how the past work fits
to create a justification of what you'll be doing
and to make your research question significant.
To get started, think about what is your topic.
And a great way to start to look for articles
that are relevant is to think of key words.
So, think about some number, five, eight,
maybe up to 10, sort of, key words and phrases
that are related to your topic.
In a class in the very near future,
we're gonna have a reference librarian come in
and help you brainstorm those.
And she's also going to show you search engines,
search indices, which will help you find articles.
One very commonly used one is Google Scholar,
where you can put your keywords in
and it will show you articles that are related to that,
just like any, you know, just like Google does,
it takes keywords and shows you links.
In this case, it specifically shows you
scholarly research articles.
And the librarian will also show you some specialty indices
that may help you find articles faster and more easy.
There's two basic types of things
that you may choose to read,
popular press and journal articles.
Sometimes a good starting point is popular press articles
and reports about your topic.
So, this can include things like magazines, newspapers,
websites, not written by scholars, written more, you know,
by journalists or by advocates.
The advantages is it tells you what others
are saying about it, what is being said sort of
in the popular discourse about the topic,
but keep in mind that while it's a good starting point,
it provides good background information
that it's not always rigorously researched.
It may be biased, it may be superficial, it may be slanted,
it may be trying to advocate for a specific point of view
and only including evidence that support that view
and omitting evidence that does not.
But, again, it can be a good starting point
just to sort of bring you up to speed.
The gold standard of scholarly research
is an article from a scholarly journal.
This is a specific account about one study
or one part of a study,
and it will walk you through an introduction.
It will itself do a lit review,
tell you about the methods they used,
the results they found, et cetera.
And usually, these are peer reviewed
and I'm gonna tell you a bit about what that means
in case you're not familiar.
So, peer review is a process that I am very familiar with,
both as an author and as a reviewer.
So, here's how it works.
Say that I have done a research project
and I write up a paper that I hope
to go into one of these journals.
I send it to the journal and the editor receives it,
and the editor reads it
and then sends it out to other reviewers.
So, one of the jobs of the editors is to read my work
and think, "Hmm, who else out there does work like this?
"Who would be a good judge of the merits of this?"
Usually, quite honestly, other professors
or folks who work in academic settings like myself.
So, those who have done work like this
and have similar interests.
The reviewers will read it, they'll judge it,
they will basically say whether it's well done or not,
and in almost every case, suggest improvements.
Tell me more about this.
I'm not quite following that, you know,
make a better case for this point, et cetera.
Then when the reviews are in, the editor sends it back
to the author, in many cases,
with the instructions to "revise and resubmit."
So, the original work isn't really ready for the journal,
but if the author makes a number of revisions based on
the recommendations of the reviewers that they,
which the author then does and resubmits it,
and sometimes it takes a few back and forth
and it is eventually, hopefully, in my case, accepted.
Sometimes the editor, the reviews will be so poor
that the editor basically concludes
that no amount of revision is going to make it worthwhile,
that it's just very flawed or not well done
or just not important or some reasons like that
and will reject it.
But in my case, hopefully, when I submit it,
it will be eventually accepted and then placed
into that journal where folks can read it.
Usually, a review is double blind,
where in the case I was speaking of,
I as the author don't know what reviewers
are going to read it and the reviewers don't know the name
of the author who wrote it.
So, it's like both sides are blind,
no one knows who the other is.
And it's so, you know, only the editor knows
and this is done so that the paper is judged solely
on its own merits and that it's well done and credible
and not that, maybe a reviewer particularly likes me
and will let you know faulty work pass,
or particularly dislikes me and rejects really good work
just 'cause they think I'm a jerk.
In the near future, again, we're gonna have
a guest in class, a reference librarian
who's going to teach you how to find articles.
Now, once you find them, here's a really good way
to read it.
So, you wanna start with the abstract,
which is the summary at the beginning,
sort of walks you through the purpose,
the basic methods, the finding.
This will both help you to decide whether
you wanna take the time to read the whole article.
And if so, it also sort of frames it for your reading
and gives you a quick overview,
so you sort of know what to expect.
So, I encourage that, you know, first sort of skim through
and jot down notes and with the major points,
then go through and take a more thorough
sort of critical detailed reading and take a lot of notes.
It's also important that if you find an article
that is really relevant to your topic,
that it can help you find other articles.
So, one of the things you can do is look
at what are the studies that the article
you're reading now cited look for,
other articles by the same author,
as well as the articles they cite.
And if you use certain search indexes
and including Google Scholar
that you can see who cited this article.
So, if you find an article from say 2015
that you think is really great and important,
you can plug it into Google Scholar
and Google Scholar will say, since that time,
here are the articles that read
and cited this article in their work.
And I call it a snowball approach that by starting with one,
and if you can find who they cite and who's cited them,
like a snowball, it can start off small,
but as it rolls down the hill it gets bigger
and bigger and bigger and bigger.
And soon, hopefully, you will have a big pile of studies
that can help form the basis of your lit review.
In general, I think it's most journal articles,
peer reviewed, that you can have some credibility
that they, or some faith that they are done well,
but a good way to look at it is to sort of think about
in these points, and I'm gonna go through each one.
So, first, looking at the research design,
is the purpose of the research clearly stated?
Do they identify a research gap
and research questions that they're going to be addressed?
Is the whole sort of research design laid out well?
Next, look at the methods.
So, sampling means, who did they talk to?
Who were their subjects?
How did they choose them?
Data collection, what methods did they use?
How did they analyze their data and how to report it?
And again, I'm gonna go over each of these
in a little bit more detail.
Later on in the course, we're gonna spend a class
thinking about sampling,
which is how the subjects are chosen.
And when you read a study, is it clear who they talked to
or where they collected data and why?
Why did they chose those subjects
or those units of analysis?
Sometimes you can think about how much
does the sample resemble the overall population?
And then, we ask, is it generalizable?
So, if you wanna know, UVM,
the whole UVM student body's view on a topic
and you ask two students,
that's not a very representative sample.
But if you get a larger and larger sample
that is drawn in ways that make sure that all aspects
of the population are represented then you can make
a better claim that the results
from this smaller group, probably closely resemble
the results of the entire population.
You can ask yourself, what methods did they use
to collect data?
Was it a survey?
Did they interview people?
Did they do observations?
And do they make sense for the research questions?
Do they, were they the right, or at least are they,
did they make a good choice of the methods they used
to get the the kind of information that they seek?
We'll learn a lot more as we go about how to analyze data,
but you can think about did they use credible techniques?
Did they use the right statistical tests,
when, if that is what they did,
and did they correctly interpret the results?
Next, how did they report out?
Are the results meaningful?
Are they tied to the research question
that they identified?
When you read it, do you have a clear understanding
of what they did, how they did it,
what was learned and what it means?
Are they upfront about any flaws or any limitations?
All studies have limitations.
It's important to be very clear and up front about that.
And if the author doesn't do that,
maybe it means they have something to hide.
And it's also a very good thing
to suggest future research needs.
Good research might answer some questions,
but it will certainly pose new ones based on what you know.
And do they lay out some future research needs
that emerge based on what they found?
So, basically you'll be able to do a much better job,
I'm hoping, of evaluating the quality
of research at the end of this class.
That is one of the major goals, to be a better consumer.
And I hope that you'll go back
and once you've taken this class
and you've sort of walked through the various methods
and options, revisit this and I think it'll make
more sense than it even does now.
Hopefully, it makes sense now, though.
I wanna talk to you a bit about internet research.
Where do you find data and how do you evaluate the quality?
So, we're gonna talk in class about
where you get information
and what search engines that you use.
You're gonna learn from our reference librarian
about a number of them,
but I wanted to talk to you mostly about
how do you evaluate the quality of claims
or research articles that you may find online
that are not in a peer reviewed article,
peer reviewed journal.
First, think of, do they advocate a clear point of view?
And I mean by that, do they have an axe to grind?
Are they just clearly cherry picking information
that confirms their point and dismissing all others?
Or do they do a good job of sort of laying out
a balanced and full account of things?
When they make a claim do they cite it?
Do they cite where does this information come from?
All good studies, and in my view, all good writing
should cite where you could get the information.
Is it up to date?
Both, was it written recently?
And if it was, are the references, the other studies
that they cite up to date?
Or did an article in 2020 only cite articles from the 1930s
or something and only out date ones,
since we've learned a lot more?
I would say that for the most part,
government data that you get from .gov
and university findings that you got from the .edu
are probably going to be more credible
than a .com or a .org.
Certainly, in recent years, I know that a lot of the data
and information on climate change
has been purged from federal databases.
So, which goes up to the first point that I think it,
in my view that they have an axe to grind
and want to sort of deny it,
but I think for the most part that government
and university generated data at these two
kind of domain names will probably
for the most part be pretty credible.
And we also wanna think about are their claims
consistent with other findings?
And not that, you know, not that it's just
confirmation bias, but certainly they, you know,
if they find something wildly out of line
with what everybody else said, you know,
that could perhaps be a cause of suspicion.
You are going to write a lit review.
And here are four possible ways
that you can organize it.
The first one is for or against a claim or an argument.
So, you could say,
climate change is real.
And here's the arguments for that.
And climate change is a hoax.
And if you can find any evidence about that,
then you could claim that, you know.
You might wanna do a little bit
less straight forward one about that,
but also next is chronological.
So, first we learned this and then we learned this
and now this is our understanding of it.
A third way is by the dimensions.
So, in a lot of my work, I'll sort of break it down.
Most of my research pertains to food and agriculture.
So, I might think about here are the economic implications.
Here are the social implications.
Here are the environmental ones.
Here are the nutritional ones.
So, sort of breaking it into these various dimensions
and then organizing the lit review based on each of these.
And last, by the views of specific actors
or stakeholders.
So, if I'm looking at a kind of food,
I might wanna think, well, what do farmers think?
What do manufacturers, what do processors,
distributors, et cetera think?
Maybe if you were looking at an issue at UVM,
say divestment from fossil fuels in our portfolio.
You might wanna look at, well,
what have, you know, students said about this?
What have faculty said?
And what have administrators say?
Organization of a lit review is extremely important
in that you do not just wanna do a laundry list.
That it's really important that you form an argument
that you curate.
So, you choose articles and carefully organize them
in a way to give it some momentum, to create an argument,
to make a point, to have a thesis statement,
and then, to identify a gap.
So, again, a good lit review
does not read like a laundry list.
It almost reads like a who-done-it,
where you come to the end and it's like,
okay, well, we know all of this stuff,
but here's a big and really important gap
that has not been touched on
and that's what I'm going to do my work on.
Here are two common mistakes that students
in the past have in my class have done.
So, they might write, this is the not a good way.
In their 2012 article in the journal, Agriculture
and Human Values, Dr. David Conner
and his colleagues found that da-la-la-la-la-la.
Just make it shorter and just give the citation.
Connor et al. 2012, found that
blah, blah, blah, blah, blah, blah.
So, you don't need the first name
and the name of the journal and all of that.
Just say the name of the authors and the year
and then the interested reader could go
and see where that is from.
The second one is to have a long list of claims
and then a bunch of citations at the end.
It's much better as you make a claim,
so you would say 0.1, Smith and Jones,
0.2, Brown and Williams and so on.
Make sure it's clear as you make claims,
which one goes with which citation.
So, if the interested reader wants to know more
and finds a claim that they think is interesting,
they could go find that article and read more.
Here are the key takeaways of this session.
First, what is a lit review?
Why do we do them?
How are they done?
Looking about various sources, particularly journal articles
and the popular press, what to look for,
the research to design and methods,
and so forth to see if they're well done.
And we also talked a bit about judging web resources.
So, here are more resources on some lit reviews
of how they may be done.
And here is also some resources
on how to do a lit review and so forth.
And I encourage you to look these up on your own time.