Integrating my online course to improve the classroom experience


Welcome everyone. Thank you all for being here. This is our seventh
year in HILT. And just for those
of you who don’t know about our
organization, we’re a university-wide organization
designed to seed innovation in teaching and learning
and to convene people across our schools to share
best practices about teaching and learning with one another. And the speaker series
is one mechanism that we use to do the
convening part of our work. We have designed this
particular mechanism with the notion of
having scholars– often in our community, but
sometimes from the outside– come in and talk
about their work and how it can improve
teaching and learning. This is our ninth one
of our speaker series. And it’s my great
pleasure to introduce Kathy to do this talk today. I think it might be the perfect
alignment of all of the things that we’ve been thinking
about with your background and what you’re working on. So Kathy is faculty co-director
of the Learning and Teaching Program at the Harvard
Graduate School of Education, and she directs the
Data Wise Project. She works with individuals
and organizations to try to get them
to work together to improve teaching
and learning, especially using
data and evidence. And that is exactly what
we’re trying to do too. And just by way
of background, she did her undergraduate in
economics at Yale and PhD in public policy here. I think it’s very
interesting to imagine that particular
educational trajectory, and then getting you to the kind
of work that you’re doing now. I also wanted to
mention that she’s going to be featured in next
Monday’s Into Practice issue. Many of you will have
seen Into Practice. If not. Sign up for this
electronic newsletter. We started meeting with
Kathy over the summer– we being Melissa,
Melissa and I– when we were initially
thinking, OK, we want to feature Kathy
in Into Practice. What are some of the ideas she
might have about low investment ways to improve
teaching and learning? And so she threw out a number
of them and then she said, well, I also do this
MOOC thing, but I’d be lying if I said that
was low investment. [LAUGHING] And then she proceeded to say,
but it was really worth it. And maybe it’s going to be one
of those investments in time that will save time
later, and certainly has changed the way I
think you’ve thought about your residential
teaching and the way you do your
residential teaching. And so that is how we ended
up with the talk today. Interestingly, this brings
together residential teaching and online teaching. And so we’ve been thinking
of this speaker series as a partnership
between us and HarvardX, and that we’re doing this,
we’re offering this talk to our community together. And so I wanted to
quickly introduce Sarah Grafman from HarvardX. She is the manager of
instructional development at HarvardX, and worked closely
with Kathy on this project. So very quickly,
because I really want to get to the good stuff,
more than 40 on campus courses are currently using or have
used HarvardX materials. That includes Architectural
Imagination at the Graduate School of Design. You can ask my colleague
Andrew over there if you have any
questions about that. Causal diagrams at the Chan
School of Public Health. Poetry is now a gen ed course. And of course,
there’s Data Wise. If anyone has any
questions about HarvardX, feel free to see me afterwards,
or Mike Kahn in the back. And I actually just got
back from three days– Erin mentioned I
worked on Data Wise, I’m actually still working
on it a little bit. Oh, ongoing! Ongoing I just got
back from three days in Prince George’s
County, Maryland where I was filming for Data
Wise, so this is just a wonderful capstone to my week. And without any further ado,
Kathy, please come on up and talk to us. Thank you, Erin, for that
very wonderful introduction. I’d literally remembered
the day HILT was announced, because I was like, at last! Those are my people! And I appreciate that you
have brought us together in so many different
ways over the– I can’t believe
it’s seven years. But it’s just–
it’s an honor to be able to talk to you about
some work that grew out of the efforts that
I feel like HILT is bringing to this school. And thank you, Sarah Sarah has
really been my right-hand woman for taking this bold step
into online learning, which is something I had not
done anything with prior. And also I just have to think
R.T. and Bill from the Teaching and Learning Lab at
Harvard, because this is what you’re going to
hear about today is very much the work of
many, a collaboration, because as Erin said, that’s
the way I like to work. And also to recognize Christina
from the Harvard Education Press, because this
whole– all of this content wouldn’t have come
to be if we didn’t have a book that we felt
we wanted to get more widely spread. So thank you, Christina
for being here. So our agenda for
today is that we’re going to start with
some introductions, and then I’m going to
just hit on a few things. Our goals, why we
even did what we did; what it was that we did
when we did what we did; some surprises that
came along the way; and a chance for some
reflections and these are your insights, your
worries, and your questions. So first I just want to get
a sense of who is in the room right now? So I know several of you, but
could you just raise a hand if you’re a faculty member? Wonderful. And if you just call
out what departments or what schools you’re at? Medical school. Medical? Sociology. Sociology? Romance languages
and anthropology. What a great thing. [INAUDIBLE] OK, so we’ve got quite
a diversity there. What about students? Wonderful And the people
who actually work on online learning sort of in a
staff capacity in some way? [LAUGHS] Trying to decide
if you wear two hats? And let me hear from the folks
who haven’t raised their hands. Raise your hand so
I know who you are. And just tell me about– well, I know who you
are– yeah, I know you. What are you involved with? I’m in the library. Library, OK. SLATE at the Kennedy School– Yes. My Alma mater. I’m at the Bok Center. Bok Center, of course. I’m in basic planning
of the school resources. Wow! That’s really cool. I really care about that. [LAUGHS] And then over
here, who do we have? I’m from the library. Library. We have a library in
the back as well, right? No. Yeah, right there in the red. Yeah. OK. All right, anyway, great
to have you all here. So I would like
for you right now– we’re going to be using these
sticky notes in a few ways. One is just write down on
one the word “why,” and then your why for why you
came here at 11 o’clock in mid-December on
a beautiful day. Let’s start with that. OK. And then as you sort of
finish your sentence, if you could just turn to
one other person and share your why’s with one
another, and I’m going to come around
and eavesdrop. So I thought I would start
with my why, which is actually the why about me
doing the work I do, because the things we’re
going to be talking about today in
terms of how to have online and residential courses. That’s just sort of
how you make it happen, but I’m animated
by the reason why I get into this in
the first place, and I’m going to be dropping
on the golden circle that Simon Sinek put
out in his TED talk. Anybody familiar with
Sinek’s Start With Why? I recommend it if you’ve
got an extra 15 minutes. So the idea is people
want to know why first, after that you
can tell them how, and then you tell them what. So that’s the order in
which I approach my work. So this guy’s face kind
of captures my why. So I believe that every single
kid in the US and in the world deserves to be held to
really high expectations for their learning. No matter what their
background/demography is, every single kid
is worthy of that, and in terms of from an
educational standpoint, that means every kid
should be able to have a really rich learning
environment that allows them to sort of grow and develop. So that’s my why. My how, I actually don’t get
to hang out with this guy. Looking at Sarah, because
she was down filming him when we first made the movie
in Delaware several years ago, but we spent a lot more
time talking to the folks that sort of underlie this
kid’s learning and all kids’ learning. What we have here is
the fourth grade team at Leasure Elementary School. Is Principal Deirdra,
assistant principal at the end, and these are the
four grade teachers, and they’re working together. Deirdra just sets aside her
Thursday and boom, boom, boom, meets with each grade level
team to talk about the data from that grade
level and how they’re going to use it to
improve instruction. And I believe that each
one of those people also deserves to be held
to really high expectations for what they can learn. Just because they became
a teacher or a principal, doesn’t mean there’s lots
of room for improvement. And in particular,
high expectations for how much they
can learn about how to work with one another. And so that’s sort of where
a lot of my effort is spent. And then the how is Data Wise,
which is the word we give for the eight-step improvement
model that was developed with faculty members, teachers,
principals from the Boston Public Schools all got together
right in the wake of No Child Left Behind, which was a piece
of legislation that made it very imperative for
people to test kids a lot, and we got people together
to say, all right, if we’re going to be
doing all this testing, what would be the
high road of taking– that you could take to use that
data to improve instruction without just trying to game
the system or, you know, bump a couple of kids up so that
your school got out of trouble? So Data Wise is our what. And if I had to sum
it up in six words, it’s about educators
collaborating so ALL– very big, there, it’s
not just for some– ALL students thrive. And that’s not all students
get great test scores. We’re talking about
thriving as we would want for any of our kids. So I’m not going to spend
much time on the what because you guys are here
to hear more about the how, but I feel like in order
to understand what we did, it’s at least important
to know that when we got that group of principals,
faculty, teachers together, after two
years, we figured out that what excellent
teachers did was they went through a
very deliberate process where they organized for
collaborative work, made sure they had time and norms for
really effective working together. They built literacy around
the kinds of assessments that they were going
to be using and didn’t assume everybody knew how to
interpret those tests just because they were
working in the building. They created a
data overview that sort of created a
sense of urgency and showed people visually why
there was a need to improve. And then, instead of going– typically, what a
lot of schools were doing was going straight from
that overview data to boom, we better reteach the
items people did poorly on and get straight to action. But the schools didn’t do
that, they went deeper. They didn’t use the
high stakes test data to figure out what was– to answer the question of
where kids were struggling, they used it as a
jumping off point to really dig into student
data, lots of different things– conversations with
students, understand what was the student
learning issue that needed to be
addressed, then they turned the mirror on themselves,
examined their own practice, and then they went to action. And so that slowing
down of the process allows, first of all, to really
pay attention to all kids. It also means bringing
in all voices of faculty in the room to sort of
be deliberately going through this process. Then you have an
action plan, you plan how you’re going to assess
it, act and assess and perhaps go around. So that’s the thing that
I’m trying to teach people. Does that makes sense? All right. And the way that we share this,
first of all, was our book. So as Christina knows well,
this book came out in 2005, and it has been the– starting then, it
became something that I was actually surprised at
how quickly it gained traction, but it showed you how
to go through each of those steps, a
chapter per step. All right. So your task as we go from
here on is to keep track– and I recommend you sort of
take three stickies in front of yourselves and put
a header on each one. So one sticky for insights
that might come up as we’re talking today. One for worries, because kind
of some of the stuff we did was a little bit risky, and if
you try to think about your why and what you’re trying to
get out of coming here, what I’m saying kind
of makes you nervous. And then your last sticky
can be your questions, and we’re going to have a
chance to sort of share these in a little while. OK, so at the time
that I first dipped my toes into online
learning, it was only 2014– I feel like it’s been a
way of life for a while, but in 2014, what we had
was this wonderful book, and we had a way
to teach people how to launch that
process that I just described in their settings. And we did it in two ways– I had a degree
program course, which was sort of a J-term module. Intensive, I get
a bunch of people into a big room at the
Ed School and figure out over the course of
a week how to get them ready to bring this
process to someone else. And then I had a Professional
Ed course in June where we brought
teachers and principals from around the world,
they came together and they learned
how to launch it. And we were actually
quite satisfied with this particular world,
except there were a couple of things that we wanted to do. So our one goal was we wanted to
really deepen the way that Data Wise was being taught. And so that week felt so short
that I had people’s attention for, and I thought, what if we
could do more during that week? What if we could make sure
that people really understood Data Wise before they
went out in the world and started to share it? And then the other thing
I wanted to do was to– oh, and so as part
of that, I wanted to be able to start those
weeks with like hitting the ground running. I wanted to have
students be invested and I’ve done some
pre-thinking about how– what I was going to
teach affected them. And I wanted to have free time,
the class time for the deeper work. The other thing
I wanted to do is I wanted to spread things
out, because I did the math and I realized that
if I had about 50, 60 people in my
January term class, and then 100-plus
in my summer class, that was going to be 160 people
a year that I was reaching. And I thought, what if I
empowered those 160 people after they had been in my care? What if they could
then have something to share back in their setting? So that was my sense of spread. And also, just the folks
that can’t come to Harvard or that we didn’t have access,
how can we create something that got them interested too? So the hope there was that
I could equip my students to freely, because I didn’t
really want to be a big marketer for having them like,
you’ve got to come to Harvard to take this class,
easily and then really– important is this one– faithfully shared. Because I was getting a little
afraid of a potential game of telephone that might
be happening in the world where I would describe
what Data Wise was, and then when somebody else
want to describe it elsewhere, that maybe something
would be lost, and I was really worried that
the thing that would be lost was my why. Because it would be just
an easy thing for people to say, oh, here’s a way
to do data because it’s important to do data. And that is not
compelling, right? I wanted to make sure
people heard from me that the reason why we
were looking at data is because it was going to let
us shine a light on what kids were learning and not learning
and figure out how to work together to address that. So I decided to jump in and
make a MOOC with HarvardX So here’s a little
screenshot from the MOOC. It was pretty quickly
we decided that it was so obvious we needed one
module for each of those steps, so that was kind of like
the organizing feature. And then I realized I really
wanted it to be self-paced. At the time that
we made our MOOC, it was a little bit less common
to have a self-paced MOOC, but I thought,
this is going to be busy teachers and
grad students, I don’t want to presume that I
knew exactly the pacing they want this content. And I knew I wanted it to
be no more than eight hours, because I wanted them to
get through the whole thing and not get kind of
bogged down anywhere. So I was like, eight hours? That’s one crazy day, one
Sunday in your pajamas! Just take the MOOC and
get it under your belt. Or that’s, you know, a couple
of months of every Monday maybe watching it with
your faculty meeting group. Just an hour together
to get through it, it seemed like that
was the right size. And so that was one of our
first thoughts was like, when we’re talking about going
online, what do we mean and what scale? And it was so liberating to
realize, that was my call. And I have never
looked back from having it be a rather small piece. And sort of another piece
of that was the book has such richness. It’s like 250 pages, all
these details about how you would actually implement this. And I didn’t want this to
just be sort of something that could substitute for the book. I wanted it to be something that
brought people in and made them just on fire to
read all 250 pages, and I thought it would be
super clear, if this is just an eight-hour experience,
that those aren’t substitutes for one another, so
it’s kind of like a reel them in kind of thing. So the structure of
our eight modules were, we would give people
content about the step– whoops– then we would have
them do a self-assessment, and then we would have
them do some practice. From a content standpoint,
they got to see sort of me and my animated swoosh, which
is what we call that arrow, and I would explain what
are all the key tasks at this particular step and
I would try to do so in as, you know, compelling
way as I could, making the sort of humdrum
stuff seem excited. But what helped was
right after that, they got into seeing an
actual school doing the work. And so because we had
filmed down at a school, they could sort
of pair that, oh, what is the step about
with how does that look in a school setting? And then they would
have self-assessments that they would do
after watching those two videos to kind of test
their understanding. Here’s a picture of the
latter inference, which is one of the mental models
that’s been enormously important in Data Wise. And this kind of
assessment I really like, because what they needed
to do was take these statements and then say, which rung
of the ladder they were on, it was kind of like a
drag-and-drop sort of thing, just to try to get people
to have one of our most important mental models be
something they already knew when I first met them in class. The other thing that they needed
to do is a self-assessment. And so the self-assessment
was actually based on an activity that had
been what I thought wildly successful in my
in-person courses, where I would pass out a
handout that looks like this, and I would give everybody
all these colored dots. And in sort of great
detail, I would go through each of
the Data Wise steps. It only took like 50
minutes, and people were putting stickers to say
whether it was happening or not happening in their setting,
then they would get up, they’d go talk to other
people, share their dots, and it kind of got
people to realize, wow, some of this stuff we
were already doing, so we’ve got strengths
we can build on. And now I understand
why we’re not making progress, because there
are spots along this arrow– it was always different
for different people– where we’re kind of
dead in the water. So that in-person
activity I wanted to have to somehow mimic
that in the online space. So what we ended up doing with
enormous help from our teaching and learning lab
and HarvardX folks was to give a self-assessment
that in the end ended up sort of when you finished
your eight-hour course, you had this lit up
swoosh that showed you where you were with the steps. And what was nice–
and I hadn’t expected– with the MOOC is, I
didn’t make people put 32 different stickers
down, because that seemed like overkill, so
we did one per step, but within, say,
step one, there’s a lot of things you
need to do, and they were able to be very specific
about which of those they’re doing and not
doing, which allowed us to have orange and
things that didn’t exist in my in-person piece. But what was fabulous
about this is I required the eight-hour course
for my residential students. Like, get it done, show up
with a printout of your arrow– or some of them,
you know, had it on their phones or whatever–
and be ready to go, which meant that the whole
morning of my first class was sort of done for
me when we showed up. And they were walking around
looking at each other’s arrows, which were much more nuanced
than the paper ones I had used, and before we even
hit lunchtime, people were talking
about how what we were about to learn that
week connected to them, and I feel like that got
a level of investment in our group that was
particularly powerful. From a practice standpoint,
there’s a lot of things that we do in Data Wise that
get people to really hold themselves back from
making inferences that aren’t supported by evidence. And so one of the
things we do is try to get people used to saying
some incredibly obvious things about what they see in a piece
of student work or in a chart, and that took a lot of
class time in my– the way I used to teach. What we did in the MOOC
was we gave people– for each of the steps
we gave them something. So an essay about an
elephant, perhaps, was we what we did at step
four, dig into student data, and the people needed
to read it, come up with what they noticed
and then wondered, and then they got to watch our
team doing a protocol where we were noticing and
wondering about the same data, and then had sort of an
online discussion about how did what they saw
differ from what we saw, and what did they
notice about the way that our team interacted
with one another? And so again, that was another
thing that I actually had– I had people practice
doing that in my setting, but I had never
like had a fishbowl and showed people
what a sort of– we kind of call it like a
low inference conversation would look like, and
this meant people got to show up in my
class already knowing what I meant about that. So that was very
much around trying to deepen so that the
people who came to my class would have a really good
sense of what the process was and why it mattered to them
before we even got started. What’s been wonderful
is I freed up hours that now I can use
for whatever I want, and one of the biggest
important pieces that we’re using that time for
is helping people to understand their why
behind why they do this work, and understand
how to communicate to others the importance
of pursuing equity in the educational context. And that had been
kind of a nice-to-have that I wished I could
have done more of, and instead, that much
more nuanced conversation can happen in-person with all
the folks that are with us. But in terms of spread, I
want to talk a little bit about what happened
with equipping students. So this is– what happened is
that people did end up sharing this with their settings. Of course, HarvardX
did an amazing job of trumpeting that
the course existed, so very quickly I realized
that we were going way beyond my residential numbers. So that’s around the US. Around the world we’ve
had 20,000 enrolled, so the spread is way beyond what
my 160 times however many years it was going to be. And you know, lots of
different countries represented, I would say outside
of English-speaking countries, we’ve got a lot of
Spanish-speaking countries as well and Brazil. And so that’s sort of
giving me a sense of, wow, this content
is now out there. And we’ve also
been able to think about how does the
MOOC sort of fit into what used to be sort
of the Data Wise world here of, like, we’ve
got these two courses and we’ve got a book. So what we’re able to do is
to back up from our launching and have an
opportunity for people to explore what Data Wise
is about and to self-assess, and that sort of– the MOOC
sort of gave this pre thing, but it also enabled me to
go beyond where I had been. So we also have a course
now in my professional ed for integrating Data Wise,
which is because I now feel comfortable that
when people go back to their settings, they’ve
got sort of like, you know, from the horse’s mouth
what Data Wise is about. And they can share it, we
have a four-month course where we have virtual
coaches to help people to actually integrate
Data Wise in their setting. And then in order to
do that, we needed to develop coaching
cadre, so we now have certified Data Wise
coaches that definitely spend some time with me
on campus, but they become people who
are either coaching Data Wise in their
settings or they’re helping with the online
coaching that happens. And so we have a coach module
for degree program students who want to become coaches
or certification people, and now we’ve got a network. And so we have a research
practicum course that basically reaches back out to the people
who are doing this in the field and has our doctoral
students sort of studying them and figuring out,
how do we feed back? How this is being received
in the world, into what we’re doing, and then
the people can be in a Data Wise coach
network who take our more elaborate courses. So I don’t want to– it’s
not actually the case that if you make a MOOC,
all of this will happen. So I don’t want you
to think that that’s what I’m trying to say, but it
made the possibility so real, because we knew there
was a way to get this in the hands of lots of people. So I’m going to tell you
about a couple of surprises and then let you
guys sort of think how this is landing for you. One thing was, there was this
word that HarvardX was saying a lot, and I think
HILT was talking a lot about repurposing of assets. I know TLL was talking
about it at school, but it’s been really
interesting to me how I can use this
in different places. So I built it as eight hours
before they even show up. But now in other classes
that I’m teaching, or if I’m teaching say a
three-hour workshop either here or somewhere else, I can click
into any one of these videos. So I just made a kind of
a cheat sheet for myself, and if I wanted
to show, what does looking at– examining your own
practice at the Delaware School look like, I can click
into that and I’ve got a five-minute video that I
can share with a large audience and explain what I mean. So that’s been really useful
to have those little pieces. The version 3.0, I didn’t
know that we were going to be able to
continuously improve, of course, on continuous
improvement, but we have. And as Sarah mentioned,
I was all excited that we were able to show
what it looked like when this was happening in a school. Sarah was down for
finding out, what does it look like
if this happens in a district of 208 schools? And so now we’ve got
that added to the MOOC as for people who have more
of a system level hat on. Discussion forum blues. I want to say that I pictured
there would be more rich back-and-forth conversations
among people on the platform than there were, and
I think that there– I would say that the actual
platform hasn’t caught up with our imaginations about how
we could get people to interact in a self-paced course
across space and time, but I’m hoping
there’ll be a version 4.0 where that can happen. And then I also was sort
of surprised in some way, pleasantly in some way not
about the book-MOOC trade-off. So the pleasant thing is just
checked in with Christina, and she says that, this has not
cannibalize sales of our book. Book sales are
still super strong. The less pleasant thing
is in my course emails, I sometimes have
students saying, it seemed like you made
us take the whole MOOC and then you made us read
the whole book and that was sort of a lot. And I worry because
they say, you know, I think just the MOOC
would have been enough, And that was my
fear, was that this would turn into some
watered-down Data Wise, where in
eight hours people could think they were doing it. So that’s kind of a thing
that I’ve got to think about and I wonder if that’s a
concern for some of you about when you put your
information out there, are people going to want sort
of like this 21st century Twitter-sized version
of what you’re doing? But mercifully the book sales
are still holding strong. So I don’t know
if people have had a chance– yes,
nicely done, I see there’s been some writing
of insights, worries, and questions. I’m going to give you sort
of just two minutes of silent think time to sort of look
through your worries and stuff and then give you a
chance to sort of get up at the boards in small groups
and kind of talk about that. So two minutes to
just figure out what you’re going
to want to hash out about what you just heard. –activity. These very Data Wise
people sort of up and them and adding to conversation
around materials, so I appreciate you
testing that out. What it does it it
guarantees that I know there’s questions
from the room now, right? So I would love to hear from– what is on your mind? I just have a fairly basic
question about Data Wise and less about MOOC-book course. To what extent is your Data
Wise model used and focused primarily on K-12
versus higher ed? And if there are higher
ed uses, what are they? That may be off-topic for
this particular session, so– OK, I’ll give a
short answer to that and we can talk more after. So yes, so higher ed is also– there’s nothing about
those eight steps that makes it have to be K-12. We started it with K-12 but
the take-up we’re having is among especially teacher
preparation programs that want to know
how to get better at teaching their educators
because they’re sort of in this particular world. But even within the Harvard
Graduate School of Education, there’s certain
pockets of things where we’re kind of taking
this eight-step approach to our work. So let let’s talk more about
what you might have in mind. The process is very
generic, as you can see, so it’s easy to apply. The microphone please. I’m just curious about
it, because your project seems to be about
assessment, so how do you assess the people who
take the book course? Whether they actually
faithfully implement the principles or the
approach that you advocate? So whether they implemented
it in their own schools and settings? Right. So for example, do they need to
submit something to you so you can interpret or– basically I’m just
wondering, how do you make sure that your
message or– yeah, the approach got implemented faithfully? I don’t. I don’t, and I just– I feel like I have to
put that out there. If you’re going to do an
eight-hour introduction course, the thing I can assess
for, which I do, is, do you understand– are people understanding
the importance of making low
inference observations, and are they capable
of distinguishing between a low inference and
a high inference statement? That I can assess for. I can assess for, are
they able to see a team and understand the
particular moves that people within that group are making to
make sure all voices get heard? Those kinds of things
I can assess for, but I have no idea how people
are doing in the world. And the thing that makes
me feel OK about that is, when I was
only selling books, I had no idea what people were
doing with the books either. So you do have to be at
peace that it’s like– that it’s part of what you do,
but that sort of elaborate set of courses, when we’re at the
level of when I’m certifying a coach, oh yeah, they submit
a 30-page report that explains how did they do it in their
setting, what did they do? But for a MOOC, it’s like– Yes? Back, just grab the
microphone yeah. My question is about how long
it took you to create this? The videos? You have the equipment, what
was the process for that? I mean, I just have to
take a moment to just say, HarvardX was extraordinary
to work with. Like something like equipment? I don’t know. Something like, you know,
what’s the filming schedule that’s happening in Delaware? Like, I went down and did a
scouting to sort of figure out what’s the story
we needed to tell, and then Sarah and her team
went down and actually made it happen on-site. So Sarah, from the very
beginning that we started, we had a few ideas
I think in, like– was it 12 months total
to get the first one out? That sounds about right. Maybe 18 if we start with ideas. Mm, yeah, 12-to-18, depending
on when we first met. Early 2014 and we launched
in spring of 2015. No. Is that right? Yeah. Yeah. So I think– Yeah. It was. Yeah, It was a little
bit over a year. And there were
moments where it took an unbelievable
amount of my time, so I’m just going
to tell you that. Like, when I had to– so
for a five-minute video, I would be in the studio for
like two hours getting the five minutes that I had to say. And just because of
the way I’m oriented, I wanted to see like the
rough cut of the video and have something to say
about how things were done. I wanted to talk
about, like, what were going to be the racial
backgrounds of the people that the artist drew to
have– doing Data Wise when I was talking about it,
you know what I mean? So a part of it
has to do with how involved you want
to be in the work, but there are moments where
it felt like it was a lot. This is very inspiring. I’m in the process of
preparing a new course, which is a service learning
course here at Harvard. I’m working on a short
pre-recording lectures, which I’m doing very simply
over campus just to have– just to save some time, Yes. As you do as well. My question to you
is, if you were in my role, what would you– is it anything that you could
advise me to think about as I’m running my course
for the first time if I would in the future like
to prepare it into the MOOC? So I am thinking about
the prerecorded lectures. I’m hoping that maybe
part of the class activity could be recorded as
well, because it’s going to be very engaging
students with their own roles and presentations. So that I could create
something like you did with the case studies. Yes! So I’m just asking you,
is there anything else that you think that I should
have in mind if in the future I would like to turn
this course into a MOOC? As I’m for the first
time running the course, if there are other
things that I can already set the wheels in motion
that would help me later on in the way? Yeah, I would say that the
two things that leap to mind are sort of paying
attention to what are the deep understandings
that you want people to have, and being clear
about what you think could be conveyed not
necessarily through experience. So I feel like trying right– at the beginning of this
course to figure out what those are, and then
looking back and figuring out that’s what they were. But then the other
thing I would say is, think about what are your
most successful in-person experiences and what
makes them successful? So at first when I started
talking about this, I’m like, well, you know,
we’re not going to be able to do my favorite
stoplight protocol, that’s not going to
work online, or I don’t know how we’re going
to get people to understand the ladder of inference
because, you know, I need that to be a very
kind of physical thing. And it’s amazing– if you’ve got
a great instructional designer working with you and if
you can talk about what do you do that’s crazy
successful in your class that gets people, you know, either
excited or really engaged, a great instructional designer
will help you kind of pull out how to make that more virtual. So don’t fear that you
have to give it all up. You were talking
about having the PhD student to go back
to the settings and do assessment
and give feedback. My question is, how
do we make sure, like, we give timely
feedback when the teachers or implementers have questions
that need to be addressed in a short time? What do we do? Or do they approach you? So it’s interesting. The settings where
people are going– where I’m sending my
doctoral students to, either virtually or in-person
see what’s happening someplace, those are places that
went way beyond the MOOC. So there are places that
started with the MOOC but then sent a team
of teachers up here and then took a
four-month virtual course, and it was in that
four-month virtual course that most of the
feedback happened. And so what was
really important was to standardize the
format in which they told their stories of what
was happening in the field. So we have this thing that’s– we call it a Data Wise
journey, but essentially it’s a PowerPoint deck that has
sort of that swoosh arrow, and then for step one it
says, what did you do? What’s the evidence
of what happened? What do you reflect on it? And we have our
virtual TFs, they’re looking through those journeys
and providing feedback as each step unfolds
about what did you do, what is the evidence
and how it goes. Because otherwise it was just
too overwhelming to think, like, you know, we’re trying to
improve in our 2,000-kid high school and are we
doing a good job? Like, I don’t know. So you have to sort
of put it on the site to give you information in a
way that your trained people are prepared to give the feedback
that will move them along. But to me, that’s
very much a question about my four-month
online course that people do
have to pay for has a much different model
in terms of the feedback, but it draws on the MOOC–
it couldn’t actually happen if we didn’t
have the MOOC because I need all
that content somewhere, and then I could really use
Brainspace to think about, and then what is our feedback
model look like when people are willing and able
to pay to receive that feedback from our team? I feel like I’m at the Oscars
every time I hold one of these. So I love the breadth
you were talking about. You know, scale and people
bringing back the learning to their communities and sharing
out the experience via the MOOC to some degree. Do you qualify those that sort
of additional sort of ring of attendees, if you will? That people who
have not attended on-campus or
participated specifically in enrollment for
a non-site course? But those people who go
back to their districts and share it out with
people, is there something as just sort of
functional as the login having to be an edu
address or having– I guess that wouldn’t make
sense in K through 12, but whatever the equivalent is. So– So you mean do I restrict
who can take the MOOC? Yes, is there any
gate-keeping, especially even since you have
a forum in terms of managing that community? One of the ways to do that
is by having your requirement for signing up, you
know, you must be and administrator,
or you must be, you know, a K
through 12 teacher. No we don’t, and we
made our mistakes. It’s just anybody
who wants to take it. We do have sort of
light facilitation where if somebody were saying
something really offensive, that would be caught, right? So that’s not happening. But it was really
important to keep it wide, and what was interesting
was, I sort of thought we made
it for educators, but you hear people who work in
all levels of different kinds of government that
might just want to know how to look at data better. The international participation
has been fascinating, like to see who is
drawn to the course, and I think if I had had
any gate-keeping on it, I might be keeping
people out who seem to be getting a lot out of it. We’re at noon, Erin! We’re at noon. The time? Yeah. I can’t believe how much
you packed in this hour. [LAUGHTER] That was
really impressive to me, and I want to thank
you all for being here. I want to thank Kathy–
yes, for helping– [APPLAUSE]

Leave a Reply

Your email address will not be published. Required fields are marked *