-: All right, so we're feeling data literate, hopefully,
and it's time to start thinking
about doing some data analytics.
And there's a little bit more prep work,
a little bit more thinking to go through
before we do that.
Number one,
you gotta think about
whether you're doing descriptive, predictive,
or prescriptive analytics.
So let's define those.
Descriptive analytics is describing what has happened.
What happens, okay?
In other words, it's looking at the past existing data.
99% of the time, that's probably what you do.
You look at existing data to figure out what happened.
Okay, predictive analytics, as the word would imply
is making predictions about the future.
Now, you can only do that
based on that descriptive analytics, right?
So we understand the past
and the patterns of what has happened,
then we can predict the future.
Now, it's worth noting predictive analytics is specific.
It's not this happened in the past,
therefore it's gonna keep happening.
It's more like this happened,
but you know,
maybe it's gonna go up by 4.7% next year, right?
So specific ideas, otherwise it's not really analytics.
Then there's also prescriptive analytics,
which is when we make recommendations,
prescriptions to alter that future.
So if we understand the past,
we make predictions about the future,
we recommend changes,
and then we can also then by the way
re predict what's gonna happen,
based on these changes we expect it
to change by 47.2, blah blah, blahs, right?
So these are the three types
of analytics we're doing,
generally, like I said,
you're probably doing descriptive analytics,
that's all good,
but I wanna share a really interesting story
about predictive analytics.
Target, years ago, famous case,
they noticed looking at their data,
like this is like data science
digging deep into like bazillions of transactions,
and they noticed that women were buying,
all of a sudden outta outta nowhere,
a bunch of unscented lotion.
And that this isn't like universally,
but certain women would buy
a lot of unscented lotion all of a sudden
and then a couple, few months later,
they would start buying diapers, and baby bottles,
and all kinds of things like this.
So, logical conclusion was, ah, interesting,
if we notice women buying a bunch of unscented lotion,
which they haven't done before,
that means that they're pregnant,
and specifically that they were
in their third trimester of pregnancy.
And lo and behold, then what they started to do,
was they started to market baby products to these women
during those couple, few months.
So they made this predictive analytics,
understanding, and prescriptive analytics,
let's market baby stuff for them.
And of course, I'm sure they predicted how many profit,
how much profits they would make.
But here's where it got interesting.
There was a 16 year old girl in Minnesota or somewhere
and these flyers started coming to the house
with her name on it, selling her baby products.
And the girl's father freaked out,
called up Target, screaming at people,
why are you sending this stuff to my daughter,
what the hell's going on here?
Turns out she was pregnant, she hadn't told him, okay,
so the predictive analytics got it right,
but of course it introduce introduces some, you know,
good conversation to have about ethics
and all kinds of things we might want to do
with our data privacy issues, et cetera.
So great example, although troubling at the same time.
So data analytics is steps, methodologies,
things you need to do,
but just like data literacy,
it's about critical thinking,
like I talked about before,
it's about thinking very carefully about what you're doing,
how you're gonna do it, why you're even doing it.
And what it's really about,
I think I've already said this before,
it's about asking questions.
What questions do you want your data to answer?
What questions do you need to answer
to do what you need to do with what, your data, okay?
And it's often about forming explicit hypotheses.
Not always, sometimes just exploring a data set
and wondering what's gonna pop out of it.
Data science often is like that
but very frequently you have specific hypotheses.
Hey, we notice this is happening in the business,
or in in science, or whatever the case may be.
We think that this might be causing it.
Then we test that hypothesis.
Either way, you're testing your data,
you're as answering questions with your data.
So you gotta form questions to ask of your data.
And you wanna start,
we wanna start with very broad questions, okay?
Really high level 30,000 foot view questions,
but you narrow in on those details,
and every question might go a little bit deeper,
a little bit deeper, a little bit deeper,
and the deeper you go,
A, it's easier to identify specific variables,
specific things you're looking at,
and more nuanced answers can come out of your data
as part of that process.
You're continuing to ask who, what, when, where, why,
all along the way
of these different fields in your data.
And you should also be asking yourself,
this is more data literacy hat,
what am I missing?
What's wrong here?
Be a little bit paranoid, like I mentioned earlier.
But we talked about this earlier,
you asked questions like, is it good or bad,
compared to what,
should the rate be used here versus the values,
et cetera, et cetera, et cetera.
Now, as you're doing this, as you're analyzing data,
what I recommend you do,
is you think about using the Toyota 5 Why's process.
So Toyota, many, many, many decades ago,
and I'm talking about
the founder of the Toyota Motor Company,
whose name was Toyota, came up with the 5 Why's.
The basic idea is you should ask why is this happening,
of any problem,
anything you're investigating.
The answer to that why
usually should lead to other questions, right?
And you should ask why again, why is that happening?
Okay, why is it this way?
Well, why, tell me why in more detail.
By the time you get to the fifth why,
the idea was,
you'll get to the actual underlying cause
rather than that surface answer.
And so you can think about it
as the 5 why's or the 5 FUs.
And I don't mean that F you, I mean follow ups.
You should be continually
following up with your questions
and eventually you get to the ground level
where you really find the great insights in your data.
So we're gonna do an exercise, okay?
And the exercise is this.
We have a data set, which is an IMDB data set.
So IMDB is a website,
Internet Movies Database,
I think is what it stands for.
I'm sure you've all seen it.
It's, you know, essentially any movie you can look,
or any actor, or director, or whatever,
you can look it up there,
and you'll find out who was in the movie,
what other movies that person was in,
et cetera, et cetera, et cetera.
This is a massive database of of movies data.
And I found a dataset
on Kaggle of the top 1000 movies from IMDB,
and we're gonna take a look at it in a second.
And the basic idea is using this dataset,
and by the way this dataset,
undergrads taking this course,
can use this dataset.
That's the one that I mentioned earlier.
I've done some pre-analysis with this data, okay?
So if you wanna do your class project using this dataset,
if you're an undergrad, go for it, all good.
For grad students,
you need to use your own research.
So this dataset, we're gonna look at this data in a second,
but the basic idea is this,
I want you to think about how,
what questions can we ask of this data?
And then how can we turn those questions
into specific metrics, variables,
and fields in the data set,
to find the answers to our questions, okay?
That's what data analytics is all about.
So you also don't wanna forget the 5 FUs.
You're gonna notice things in the data
as you're analyzing it,
and you should be constantly asking, okay,
well that's interesting,
but what does that mean going deeper?
Okay, now let's do a couple few together to start.
All right, so let's look at the data set.
So we have, as you see here, the name of the movie.
And by the way, this is the IMDB top 1000,
which is the top 1000 movies, I think, by the,
what's called the META score here.
Okay, so the META score is like a,
merged together, aggregated, averaged out score
based on critics ratings of the movie, I believe.
So anyways, we have the name of the movie,
we have the year it was released, runtime,
we have the genre.
And in fact, you'll notice that some of these are,
it's more than one comma delimited.
We'll talk more about that.
We have the IMDB rating,
which is what people visiting the IMDB website
had given it as their rating
up to a score of 10, I believe.
We have an overview,
which is just a short text description of the movie.
We have that META score that I mentioned.
Then we have the director and then four of the actors.
And I don't know if these are in order,
I don't know why it's star 1, 2, 3, 4 exactly.
We also have the number of votes,
the number of those people visiting the website
giving it that IMDB rating.
And we also have the gross,
which is essentially the gross revenues that the movie made.
So this is our data set.
What questions might we ask of this data?
I could think of a bunch
of questions off the top of my head.
I could ask questions like, which are the better movies,
longer movies, or shorter movies?
What does runtime have to do
with quality in terms of the score?
I could ask,
are movies from different years or decades
scored higher or lower?
I could ask questions like
which genres lead to higher or lower scores?
I could ask, is the META score
and the IMDB rating two different ways
of looking at the quality of this movie?
Are those correlated?
Or is there a difference between those?
How about what directors and
or actors appear in the most popular movies?
Or how many of them appear more than once?
I could ask questions like
does the number of votes correlate
with that IMDB score?
I could ask questions about profitability,
gross or not profitability,
cause I don't know what the costs are,
but at least gross revenues
is that correlated with quality in any which way?
Those are just basic questions off the top of my head.
So easy to and, to come up with.
And I'm sure there are more nuanced questions
we could explore as well, right?
So this is what we need to do when we're analyzing data,
is ask questions of our data like those.
And by the way
we may find once we've answered those questions
that we find further questions.
And in fact, when I introduced the analysis
that I have done with this dataset,
later on you'll notice I discovered an answer
to one of those questions that I just mentioned
and it led me to further questions, okay?
So that's getting ready to do data analytics.
And before we can move on,
and we're gonna do that as another video in a second,
I have to realize that you know what, data is garbage.
It sucks.
It's always a mess, okay?
Which means we always have to clean it up.
We always, I guess shouldn't say always,
99.9999999% of the time,
you have to mess with your data
before you can actually analyze it.
So that's what we're gonna do next.