1
00:00:00,506 --> 00:00:08,946
[ Silence ]

2
00:00:09,446 --> 00:00:11,606
>> Welcome back to
Computer Science S-75.

3
00:00:11,606 --> 00:00:13,646
This is Lecture Nine,
our very last [laughter].

4
00:00:13,716 --> 00:00:15,286
It's been a pleasure
having everyone

5
00:00:15,286 --> 00:00:16,436
in the course this semester.

6
00:00:16,696 --> 00:00:18,406
So, tonight we talk
about "Scalability."

7
00:00:18,406 --> 00:00:21,196
So, we try to revisit some of
the topics that we've looked

8
00:00:21,266 --> 00:00:22,846
at earlier in the
semester and think

9
00:00:22,846 --> 00:00:25,726
about how we can deploy
applications not just

10
00:00:25,726 --> 00:00:28,276
on say virtual machine, on
your laptop, or desktop,

11
00:00:28,276 --> 00:00:29,406
as we've been doing
with the appliance.

12
00:00:29,726 --> 00:00:32,336
But how you can scale two
servers on the Internet

13
00:00:32,336 --> 00:00:34,356
and indeed multiple
servers on the Internet

14
00:00:34,356 --> 00:00:36,166
so that you can handle
hundreds, or thousands,

15
00:00:36,166 --> 00:00:39,676
or tens of thousands, or even
more than that in theory.

16
00:00:39,676 --> 00:00:41,196
So, some of the issues

17
00:00:41,196 --> 00:00:43,966
that we'll inevitably encounter
is how to go about doing this.

18
00:00:43,966 --> 00:00:46,576
So, when it comes time to put
something on the Internet,

19
00:00:46,576 --> 00:00:49,216
recall from Lecture Zero, that
we talked about web hosts.

20
00:00:49,216 --> 00:00:52,346
So, this is by no means a list
of recommendations per se,

21
00:00:52,346 --> 00:00:53,796
it's just some representative
ones

22
00:00:53,796 --> 00:00:55,016
that we happen to recommend.

23
00:00:55,106 --> 00:00:57,616
If only because the
teaching fellows

24
00:00:57,616 --> 00:01:00,416
and I have had prior experiences
with these particular vendors.

25
00:01:00,676 --> 00:01:02,756
But you can Google around and
you can see that there's many,

26
00:01:02,756 --> 00:01:04,576
many, many different
options these days.

27
00:01:04,576 --> 00:01:06,696
However, among the
takeaways hopefully

28
00:01:06,696 --> 00:01:09,376
from the summer thus
far has been what kinds

29
00:01:09,376 --> 00:01:11,346
of features should
you be looking for

30
00:01:11,346 --> 00:01:14,206
or expecting minimally in
any web hosting company

31
00:01:14,206 --> 00:01:15,026
that you might choose.

32
00:01:15,716 --> 00:01:20,136
And in fact, not all of
these even have those

33
00:01:20,136 --> 00:01:20,976
features necessarily.

34
00:01:21,516 --> 00:01:35,126
[ Inaudible Speaker ]

35
00:01:35,626 --> 00:01:36,806
Interesting, okay good.

36
00:01:36,806 --> 00:01:40,696
So, if your country or your
work or really any network

37
00:01:40,696 --> 00:01:42,656
that you happen to be on or
people that you know happen

38
00:01:42,656 --> 00:01:45,366
to be on and block access
to certain IP ranges,

39
00:01:45,726 --> 00:01:48,036
among them for instance,
GoDaddy in this case.

40
00:01:48,036 --> 00:01:49,596
YouTube is a popular
thing to block,

41
00:01:49,596 --> 00:01:51,006
Facebook is a popular
thing to block.

42
00:01:51,006 --> 00:01:54,366
That can be a sticking point,
so doing a bit of due diligence

43
00:01:54,366 --> 00:01:56,256
or testing first
could be a good thing.

44
00:01:57,046 --> 00:02:00,176
What else should you look
for in a hosting company?

45
00:02:00,726 --> 00:02:00,846
Isaac?

46
00:02:01,846 --> 00:02:01,966
>> SFTP?

47
00:02:02,176 --> 00:02:07,326
>> Good, SFTP in
contrast with what?

48
00:02:07,886 --> 00:02:07,976
>> FTP.

49
00:02:07,976 --> 00:02:08,726
>> FTP and why?

50
00:02:09,796 --> 00:02:12,176
>> Because FT -- SFTP is secure.

51
00:02:12,266 --> 00:02:12,756
>> Okay, good.

52
00:02:12,756 --> 00:02:15,076
So, the "S" literally stands for
"Secure" and what that means is

53
00:02:15,076 --> 00:02:17,206
that all of your
traffic is encrypted.

54
00:02:17,206 --> 00:02:19,316
And this maybe isn't a
big deal for the files

55
00:02:19,316 --> 00:02:20,896
that you're transferring,
because after all,

56
00:02:20,896 --> 00:02:24,096
if you're uploading like gifs,
and jpegs, and video files

57
00:02:24,096 --> 00:02:26,366
that are meant to be downloaded
by people on the web, well,

58
00:02:26,366 --> 00:02:28,066
then who really cares
of you encrypt those

59
00:02:28,066 --> 00:02:29,746
between your machine
and the server.

60
00:02:30,026 --> 00:02:32,786
But it is important to
have what data encrypted?

61
00:02:33,796 --> 00:02:34,076
Jack?

62
00:02:34,466 --> 00:02:35,386
>> Usernames and
passwords [inaudible].

63
00:02:36,116 --> 00:02:38,436
>> Exactly [laughter],
usernames and passwords, right?

64
00:02:38,436 --> 00:02:41,676
I mean that's one of the biggest
failings of something like FTP

65
00:02:41,676 --> 00:02:44,116
which granted is a fairly
dated or older protocol,

66
00:02:44,326 --> 00:02:46,946
is that it sends also your user
name and password in the clear.

67
00:02:46,946 --> 00:02:49,226
Which means anyone sniffing
wirelessly around you,

68
00:02:49,446 --> 00:02:53,016
anyone sniffing the wire network
between point A and B can see

69
00:02:53,016 --> 00:02:54,976
in the clear what your
user name and password are.

70
00:02:55,146 --> 00:02:55,236
Yeah.

71
00:02:55,636 --> 00:02:59,776
>> [Inaudible] posting companies
that offer unlimited everything

72
00:02:59,776 --> 00:03:00,756
for a really low price.

73
00:03:00,816 --> 00:03:01,286
>> Okay, good.

74
00:03:01,286 --> 00:03:05,906
>> That would be due
to a virtual hosting -

75
00:03:06,276 --> 00:03:06,466
>> Good.

76
00:03:06,466 --> 00:03:08,716
>> -- that you wanted to
implement a system that can grow

77
00:03:08,716 --> 00:03:11,726
by itself, maybe you don't want
to share the same computers,

78
00:03:12,486 --> 00:03:14,966
the same server with
many others.

79
00:03:15,156 --> 00:03:17,396
>> Good, so Dreamhost in
particular, I think we pulled

80
00:03:17,396 --> 00:03:21,006
up their feature list and it
was ridiculous how many features

81
00:03:21,006 --> 00:03:22,936
they give you, unlimited
tri-bandwidth,

82
00:03:22,936 --> 00:03:25,796
unlimited storage space,
unlimited BRAM or something

83
00:03:25,796 --> 00:03:28,826
like that and that just
can't possibly be real

84
00:03:28,826 --> 00:03:32,456
if you're only paying
like $9.95 a month.

85
00:03:32,456 --> 00:03:34,406
So there's some catch and
in general, the catch is

86
00:03:34,436 --> 00:03:37,526
that they're making that
offer to you and to a 100,

87
00:03:37,526 --> 00:03:40,606
to hundreds of other
customers all of whom might be

88
00:03:40,606 --> 00:03:43,596
on that same machine, so you're
contending now for resources.

89
00:03:43,596 --> 00:03:45,586
And the reality is they're
probably banking on the fact

90
00:03:45,586 --> 00:03:47,766
that 90 something percent of
their customers don't need

91
00:03:48,006 --> 00:03:50,736
that many resources but for
the one person or two persons

92
00:03:50,736 --> 00:03:52,546
that do, probably going the way

93
00:03:52,546 --> 00:03:54,226
of a shared host
is not necessarily

94
00:03:54,226 --> 00:03:55,926
in your best interest,
certainly not something to try

95
00:03:55,926 --> 00:03:57,836
to build a bigger business on.

96
00:03:57,836 --> 00:03:59,396
And so, there's alternatives
to thing

97
00:03:59,396 --> 00:04:01,506
like web posting
companies, there's VPSs,

98
00:04:02,096 --> 00:04:05,516
a virtual private server
that you can essentially rent

99
00:04:05,516 --> 00:04:08,386
for yourself and what's
the fundamental distinction

100
00:04:08,386 --> 00:04:10,876
between a VPS and
a shared web post?

101
00:04:12,206 --> 00:04:12,476
Axle?

102
00:04:12,776 --> 00:04:15,706
>> The VPS is like
borrowed [inaudible] Linux.

103
00:04:16,046 --> 00:04:19,366
It's a virtual O
machine running on cost

104
00:04:19,556 --> 00:04:21,716
but it's completely
its own system.

105
00:04:21,716 --> 00:04:22,376
>> Okay, good.

106
00:04:22,376 --> 00:04:23,576
Well and so to be clear,

107
00:04:23,576 --> 00:04:26,466
the operating system is largely
irrelevant in stream host

108
00:04:26,466 --> 00:04:28,686
and shared web posts could
also be running Fedora

109
00:04:28,686 --> 00:04:29,696
or any operating system

110
00:04:29,836 --> 00:04:33,936
but what's key is you get your
own copy of Fedora or Ubuntu

111
00:04:33,936 --> 00:04:35,946
or whatever operating system
they happen to be running.

112
00:04:35,946 --> 00:04:37,566
Because in the world of VPSs,

113
00:04:37,566 --> 00:04:39,896
what they do is they take
generally a super-fast server

114
00:04:39,896 --> 00:04:42,786
with lots of RAM, lots of
CPU, lots of disc space,

115
00:04:42,956 --> 00:04:44,736
and they chop it up
into the illusion

116
00:04:44,736 --> 00:04:47,076
of multiple servers using
something called the hypervisor

117
00:04:47,076 --> 00:04:51,146
or something like a product
from VMware, or Citrix,

118
00:04:51,146 --> 00:04:53,686
or other companies, and even
open source providers have

119
00:04:53,806 --> 00:04:56,576
platforms that allow you
to do this, run multiple -

120
00:04:56,576 --> 00:05:00,576
a virtual machine on a physical
machine and so, in this world,

121
00:05:00,576 --> 00:05:02,276
you're still sharing resources

122
00:05:02,556 --> 00:05:04,606
but different --
in a different way.

123
00:05:04,606 --> 00:05:09,016
You're instead, getting some
slice of hardware to yourself

124
00:05:09,016 --> 00:05:10,996
and no one else has
user accounts

125
00:05:11,186 --> 00:05:13,066
on that particular
virtual machine.

126
00:05:13,296 --> 00:05:15,326
Now, with that said, the
system administrator,

127
00:05:15,396 --> 00:05:19,196
the owners of the VPS company,
they themselves depending

128
00:05:19,196 --> 00:05:20,466
on what hypervisor
they're using,

129
00:05:20,656 --> 00:05:23,426
they might actually have
access to your virtual machine

130
00:05:23,426 --> 00:05:25,906
and to your files and frankly,
if you have physical access

131
00:05:25,906 --> 00:05:28,976
to the machine, you undoubtedly
have access to your files

132
00:05:28,976 --> 00:05:30,946
because they can always
reboot the virtual machine,

133
00:05:31,236 --> 00:05:33,186
for instance, in what's
called "single-user mode"

134
00:05:33,186 --> 00:05:35,576
or "diagnostic mode" and
at that point, it's not --

135
00:05:35,576 --> 00:05:37,516
they're not even prompted
for a root password.

136
00:05:37,516 --> 00:05:39,836
So, realize that even
when you're using a VPS,

137
00:05:39,836 --> 00:05:43,326
your data might be more
secure, more private,

138
00:05:43,486 --> 00:05:44,456
from other customers

139
00:05:44,546 --> 00:05:47,176
but definitely not the web
posting company itself.

140
00:05:47,176 --> 00:05:49,626
If you want even more privacy
than that, you're probably going

141
00:05:49,626 --> 00:05:51,936
to have to operate your
own servers that only you

142
00:05:51,936 --> 00:05:54,566
or your colleagues have
physical access to.

143
00:05:54,566 --> 00:05:57,086
So, here's just a list of some
of the popular VPS companies.

144
00:05:57,366 --> 00:06:00,076
There is one catch - in order
to get these additional features

145
00:06:00,076 --> 00:06:04,426
or these properties in a
VPS, you generally pay more.

146
00:06:04,496 --> 00:06:06,656
So, instead of $10 a
month, $20 a month,

147
00:06:06,656 --> 00:06:10,366
you're probably starting at $50
a month or something like that,

148
00:06:10,366 --> 00:06:12,396
maybe even in the hundreds
depending on how many --

149
00:06:12,646 --> 00:06:16,416
how much you want in
the way of resources.

150
00:06:16,886 --> 00:06:17,976
And toward the end of today,

151
00:06:17,976 --> 00:06:22,176
we'll talk about one
particular vendor of VPSs,

152
00:06:22,296 --> 00:06:25,796
namely "Amazon Web
Services, Amazon EC2,"

153
00:06:25,796 --> 00:06:27,156
their elastic compute cloud

154
00:06:27,446 --> 00:06:30,286
that essentially lets you
self-service and spawn

155
00:06:30,286 --> 00:06:32,926
as many virtual machines as you
want so long as you're willing

156
00:06:32,926 --> 00:06:35,406
to pay some number of
cents per minute to have

157
00:06:35,406 --> 00:06:37,256
that virtual machine
up and running.

158
00:06:37,296 --> 00:06:43,236
It's actually a wonderful way
to plan for unexpected growth

159
00:06:43,236 --> 00:06:44,796
because you can even
automate the process

160
00:06:44,796 --> 00:06:47,136
of spawning more web servers,
more database servers,

161
00:06:47,136 --> 00:06:49,806
if you happen to suddenly
get popular, even over night

162
00:06:49,906 --> 00:06:52,296
or because you've been slash
dotted or posted on reedit

163
00:06:52,296 --> 00:06:54,556
or the like, and then you
can have those machines

164
00:06:54,556 --> 00:06:57,586
automatically power off when
interest in your product

165
00:06:57,586 --> 00:07:00,166
or website has started
to subside.

166
00:07:00,676 --> 00:07:04,166
All right, so how -- suppose
you are the fortunate sufferer

167
00:07:04,166 --> 00:07:06,196
of a good problem which
is that your website all

168
00:07:06,196 --> 00:07:08,276
of a sudden is super popular

169
00:07:08,276 --> 00:07:12,036
and this website has maybe some
static web content, HTML files,

170
00:07:12,036 --> 00:07:14,996
gifs, and the like, dynamic
content like PHP code,

171
00:07:15,646 --> 00:07:18,456
maybe even some database stuff,

172
00:07:18,606 --> 00:07:21,776
maybe some database
stuff like MySQL.

173
00:07:21,906 --> 00:07:24,176
Well, how do you go
about scaling your site

174
00:07:24,396 --> 00:07:27,016
so that it can handle
more users?

175
00:07:27,016 --> 00:07:29,996
Well, the most straightforward
approach is generally what's

176
00:07:29,996 --> 00:07:32,726
called "Vertical Scaling",
vertical in the sense that --

177
00:07:32,926 --> 00:07:36,026
well, if you're running
low on RAM or you're kind

178
00:07:36,026 --> 00:07:39,186
of exhausting your available
CPU cycles or you're running low

179
00:07:39,186 --> 00:07:41,586
on disc space, what's
the easiest,

180
00:07:41,586 --> 00:07:43,336
most obvious solution
to that problem?

181
00:07:44,806 --> 00:07:45,076
Axle?

182
00:07:45,466 --> 00:07:46,946
>> Get a better processor
and more RAM.

183
00:07:47,016 --> 00:07:49,556
>> Good, get more RAM, more
processor, more disc space

184
00:07:49,556 --> 00:07:50,896
and just throw resources

185
00:07:50,896 --> 00:07:53,176
or equivalently money
at the problem.

186
00:07:53,176 --> 00:07:54,816
Unfortunately, there
is a catch here.

187
00:07:54,816 --> 00:07:58,046
There's sort of a ceiling
on what you can do why?

188
00:07:58,856 --> 00:08:04,126
Why is vertical scaling not
necessarily a full solution?

189
00:08:04,696 --> 00:08:05,826
>> Well, you can only
operate one machine so much

190
00:08:05,826 --> 00:08:08,266
after a while you
can't [inaudible].

191
00:08:08,266 --> 00:08:08,936
>> Yeah, exactly.

192
00:08:08,936 --> 00:08:10,166
There's some real
world constraints here

193
00:08:10,166 --> 00:08:14,236
where you can only buy a machine
that's maybe three gigahertz

194
00:08:14,266 --> 00:08:17,556
these days, and maybe only
has a few - a handful of -

195
00:08:17,556 --> 00:08:21,766
maybe a couple dozen CPUs
or cores but at some point,

196
00:08:22,006 --> 00:08:24,966
you're either going to exhaust
your own financial resources

197
00:08:24,966 --> 00:08:27,106
or just the state of
the art in technology

198
00:08:27,106 --> 00:08:29,686
because just the world hasn't
made a machine that has

199
00:08:29,686 --> 00:08:30,996
as many resources as you need.

200
00:08:31,166 --> 00:08:33,506
So, you need to get a
little smarter, but at least

201
00:08:33,506 --> 00:08:35,496
within here, you
have some discretion.

202
00:08:35,496 --> 00:08:39,186
So, in terms of CPUs these
days, most servers have

203
00:08:39,186 --> 00:08:43,386
at least two CPUs, sometimes
three or four or more

204
00:08:43,536 --> 00:08:46,246
and in turn, each of those CPUs
typically has multiple cores.

205
00:08:46,246 --> 00:08:49,066
In fact, most of the laptops you
guys have here these days are

206
00:08:49,066 --> 00:08:52,366
generally at least duo core,
sometimes even quad core

207
00:08:52,556 --> 00:08:55,886
which means that you effectively
have the equivalent of four CPUs

208
00:08:56,096 --> 00:08:58,776
or four brains inside of your
computer even though they're all

209
00:08:58,776 --> 00:09:01,386
inside of the same
chip essentially.

210
00:09:01,386 --> 00:09:02,616
What does that mean concretely?

211
00:09:02,616 --> 00:09:05,366
It means if you have a quad
core machine, you can --

212
00:09:05,366 --> 00:09:07,966
your computer can literally
do four things at once.

213
00:09:07,966 --> 00:09:10,516
Whereas, in yesteryear,
when you had single cores,

214
00:09:10,576 --> 00:09:13,976
single CPU machines, they could
only do one thing at a time

215
00:09:14,136 --> 00:09:17,016
and even though we
humans seem to think

216
00:09:17,236 --> 00:09:19,326
that you're simultaneously
printing something,

217
00:09:19,326 --> 00:09:21,116
you're pulling up a Google
map, you're getting email

218
00:09:21,116 --> 00:09:23,386
and all this stuff seems to
be happening simultaneously,

219
00:09:23,696 --> 00:09:26,726
the reality is the operating
system is scheduling each

220
00:09:26,726 --> 00:09:29,266
of those programs to
get just a split second

221
00:09:29,326 --> 00:09:31,936
of CPU time before
giving another program,

222
00:09:31,936 --> 00:09:34,226
then another program a
split second of CPU time,

223
00:09:34,496 --> 00:09:38,356
and we humans are just so slow
relative to today's processors,

224
00:09:38,606 --> 00:09:39,496
that we don't even notice

225
00:09:39,596 --> 00:09:41,336
that things are actually
happening serially

226
00:09:41,476 --> 00:09:43,546
as opposed to in parallel.

227
00:09:43,546 --> 00:09:46,546
But when you actually have quad
cores, especially in a server,

228
00:09:46,766 --> 00:09:49,736
that means whereas in yesteryear
with a single core machine,

229
00:09:49,736 --> 00:09:52,436
you could handle one
web request at a time.

230
00:09:52,656 --> 00:09:55,656
Now, for instance, you can
handle at least four at a time,

231
00:09:55,656 --> 00:09:57,786
truly in parallel and even then,

232
00:09:57,786 --> 00:10:00,086
a server will typically spawn
what are called multiple

233
00:10:00,086 --> 00:10:00,976
processes or multiple threads.

234
00:10:01,076 --> 00:10:05,036
So, in reality, you can at
least give the impression

235
00:10:05,036 --> 00:10:06,436
that you're handling many more

236
00:10:06,486 --> 00:10:08,666
than even four requests
per second.

237
00:10:08,666 --> 00:10:11,726
So in short, machines these
days have gotten more and more

238
00:10:11,726 --> 00:10:15,136
and more CPUs as well
as more cores and

239
00:10:15,136 --> 00:10:18,096
yet the funny thing here is that
we humans also aren't very good

240
00:10:18,096 --> 00:10:21,026
at figuring out what to do with
all of this available hardware.

241
00:10:21,026 --> 00:10:23,136
Right? Most of you, even if
you have a dual core machine,

242
00:10:23,136 --> 00:10:26,236
you don't really need a dual
core machine to check your mail

243
00:10:26,236 --> 00:10:28,176
or to like write an
essay for school.

244
00:10:28,176 --> 00:10:30,176
You were able to do
that five, 10 years ago

245
00:10:30,386 --> 00:10:32,536
with far fewer computational
resources.

246
00:10:32,796 --> 00:10:35,656
Now, in fairness, there's
bloat in software and MAC OS

247
00:10:35,696 --> 00:10:37,976
and Windows and Office are
just getting bigger and bigger,

248
00:10:37,976 --> 00:10:39,576
so we're using those resources.

249
00:10:39,826 --> 00:10:42,656
But one of the really
nice results

250
00:10:42,736 --> 00:10:44,346
of this trend toward more

251
00:10:44,346 --> 00:10:46,406
and more computational
resources is that,

252
00:10:46,406 --> 00:10:49,556
the world has been able all the
more easily to start chopping

253
00:10:49,556 --> 00:10:51,876
up bigger servers
into smaller VPSs.

254
00:10:51,876 --> 00:10:54,796
And indeed, that's how Amazon
and other cloud [inaudible]

255
00:10:54,976 --> 00:10:57,736
so to speak are able
to provide people

256
00:10:57,736 --> 00:11:00,916
with the self-service capability
as we'll discuss a bit later.

257
00:11:01,286 --> 00:11:03,606
So, within these things there
are a few things you have

258
00:11:03,606 --> 00:11:04,866
discretion over.

259
00:11:05,186 --> 00:11:07,456
If you've ever built a
computer, you might be familiar

260
00:11:07,456 --> 00:11:13,506
with parallel ATA, or IDE,
or SATA, or SAS anyone?

261
00:11:14,156 --> 00:11:15,336
Axle, what do these refer to?

262
00:11:15,666 --> 00:11:17,466
>> SATA has to do
with hard drives.

263
00:11:17,716 --> 00:11:18,266
>> Okay, good.

264
00:11:18,266 --> 00:11:19,526
So, SATA has to do
with hard drives --

265
00:11:19,526 --> 00:11:21,776
in fact, all three of those
have to do with hard drives.

266
00:11:21,776 --> 00:11:23,596
Years ago, parallel ATA

267
00:11:23,596 --> 00:11:26,466
or IDE hard drives were
very much in vogue.

268
00:11:26,466 --> 00:11:28,826
You might still have them in
older computers these days,

269
00:11:28,826 --> 00:11:30,836
desktop computers pretty much,

270
00:11:30,876 --> 00:11:34,756
but you wouldn't buy a new
parallel ATA drive these days.

271
00:11:34,756 --> 00:11:36,746
Instead you'd most
likely get a SATA drive

272
00:11:37,046 --> 00:11:40,066
where they're 3.5 inch for a
desktop or two and a half inch

273
00:11:40,156 --> 00:11:44,556
for a laptop and if you have
servers or you have lots

274
00:11:44,556 --> 00:11:47,006
of money and a fancy
desktop computer,

275
00:11:47,006 --> 00:11:48,996
you can go with the SAS drive.

276
00:11:49,076 --> 00:11:50,996
SAS is Serial Attached SCSI

277
00:11:51,166 --> 00:11:53,596
and this really just
boils down to faster.

278
00:11:53,596 --> 00:11:55,206
For instance, whereas
parallel ATA

279
00:11:55,206 --> 00:12:00,736
and SATA drives typically
ran at 7200 rpms per minute,

280
00:12:00,846 --> 00:12:03,046
revolutions per minute,
SAS drives,

281
00:12:03,296 --> 00:12:05,326
anyone know what they
typically spin at?

282
00:12:05,856 --> 00:12:10,286
And for those unfamiliar, inside
of a mechanical hard drive,

283
00:12:10,326 --> 00:12:13,916
there is a one or more metal
platters that literally spins,

284
00:12:13,916 --> 00:12:15,976
much like an old-school
record player

285
00:12:16,366 --> 00:12:17,746
where the bits are now stored.

286
00:12:18,626 --> 00:12:20,916
What speed does a
SAS drive spin?

287
00:12:20,916 --> 00:12:22,666
It's more than 7200 rpm.

288
00:12:22,846 --> 00:12:23,176
Axle?

289
00:12:23,346 --> 00:12:24,956
>> Is it 15,000?

290
00:12:24,956 --> 00:12:28,066
>> Yeah, so 15,000 is where
they would typically perform,

291
00:12:28,066 --> 00:12:29,766
sometimes 10,000 but 15,000.

292
00:12:29,766 --> 00:12:32,426
So, just twice as fast so
that alone gives you a bit

293
00:12:32,426 --> 00:12:33,076
of a speed bump.

294
00:12:33,076 --> 00:12:35,926
Of course, it comes at a
price literally more money

295
00:12:36,226 --> 00:12:38,046
but that's one way of
speeding things up.

296
00:12:38,046 --> 00:12:42,526
So, oftentimes what people
will do is for a given website

297
00:12:42,526 --> 00:12:45,566
that they're creating, if it
has a database, databases tend

298
00:12:45,566 --> 00:12:47,096
to write to disc quite a bit.

299
00:12:47,096 --> 00:12:49,856
Right? Every Facebook update
requires writing to disc

300
00:12:49,856 --> 00:12:52,196
and then reading back
out some number of times.

301
00:12:52,486 --> 00:12:55,256
So, really where you might
be touching disc a whole lot,

302
00:12:55,556 --> 00:12:58,906
people can throw things like
SAS drives in their database

303
00:12:58,906 --> 00:13:02,126
so that their data can be read
or written even more quickly

304
00:13:02,126 --> 00:13:06,156
and what's even faster than
mechanical drives these days?

305
00:13:06,766 --> 00:13:07,036
Axle?

306
00:13:07,776 --> 00:13:08,596
>> Solid state drives?

307
00:13:08,596 --> 00:13:11,596
>> Yeah, solid state drives,
SSDs, which have no moving parts

308
00:13:11,596 --> 00:13:14,026
and as a result,
electrically perform much better

309
00:13:14,336 --> 00:13:15,676
than mechanical drives.

310
00:13:15,716 --> 00:13:18,486
But those too, cost more money
and they tend to be smaller

311
00:13:18,486 --> 00:13:20,086
in size so whereas you can buy

312
00:13:20,476 --> 00:13:23,756
like a four terabyte
SATA drive these days,

313
00:13:23,886 --> 00:13:25,666
three and a half inch
for your desktop,

314
00:13:25,916 --> 00:13:31,106
you can buy maximally a
768 gigabyte SSD these days

315
00:13:31,106 --> 00:13:32,716
for a lot more money typically.

316
00:13:33,836 --> 00:13:34,206
All right.

317
00:13:34,666 --> 00:13:40,466
So, let's skip -- let's
skip RAID for now.

318
00:13:41,216 --> 00:13:42,426
So, horizontal scaling --

319
00:13:42,526 --> 00:13:44,426
so this is in contrast with
what we just discussed,

320
00:13:44,426 --> 00:13:47,636
throwing money and throwing
more of everything at a problem.

321
00:13:47,846 --> 00:13:50,706
Horizontal scaling is
sort of accepting the fact

322
00:13:50,706 --> 00:13:52,406
that there's going to
be a ceiling eventually,

323
00:13:52,636 --> 00:13:54,416
so why don't we instead
architect our system

324
00:13:54,586 --> 00:13:56,716
in such a way that we're
not going to hit that.

325
00:13:57,216 --> 00:14:01,016
Rather, we can kind of stay
below it by even using not state

326
00:14:01,016 --> 00:14:03,586
of the art hardware and the
most expensive stuff we can buy,

327
00:14:03,846 --> 00:14:04,726
but cheaper hardware.

328
00:14:04,726 --> 00:14:06,516
Servers that might
be a few years old

329
00:14:06,516 --> 00:14:09,916
or at least are not
the top of the line,

330
00:14:09,916 --> 00:14:11,476
so that they'll be
less expensive.

331
00:14:11,476 --> 00:14:14,766
So, rather than get few or
one really good machine,

332
00:14:14,996 --> 00:14:17,036
why don't we get
a bunch of slower

333
00:14:17,216 --> 00:14:19,726
or at least cheaper
machines instead,

334
00:14:20,146 --> 00:14:21,376
plural number of machines?

335
00:14:21,596 --> 00:14:23,906
So, this is just a picture of a
data center which is just meant

336
00:14:23,906 --> 00:14:26,056
to conjure up the idea
of scaling horizontally

337
00:14:26,156 --> 00:14:28,466
and actually using
multiple servers to build

338
00:14:28,466 --> 00:14:32,886
out your topology but what
does this actually mean?

339
00:14:32,886 --> 00:14:36,456
Well, if you have a whole
bunch of servers now,

340
00:14:36,746 --> 00:14:40,426
instead of just one, how -
what's the relationship now

341
00:14:40,426 --> 00:14:44,566
with Lecture Zero where we
talked about HTTP and DNS?

342
00:14:45,446 --> 00:14:45,616
All right.

343
00:14:45,616 --> 00:14:48,986
The world was very simple a few
weeks ago when you had a server

344
00:14:48,986 --> 00:14:50,166
and it had an IP address

345
00:14:50,166 --> 00:14:51,846
and that IP address
might have a domain name

346
00:14:51,846 --> 00:14:54,556
or host name associated with
it and we told that story

347
00:14:54,556 --> 00:14:56,836
of what happens when you type
in something dot com enter

348
00:14:56,836 --> 00:15:00,346
on your laptop and you get back
the pages on that single server.

349
00:15:00,346 --> 00:15:01,576
But now, we have a problem

350
00:15:01,576 --> 00:15:04,446
if we have a whole aisle's
worth of web servers.

351
00:15:04,446 --> 00:15:04,836
Axle?

352
00:15:05,076 --> 00:15:07,516
>> Well, you're going to have to
have a way to treat each request

353
00:15:07,516 --> 00:15:10,566
so it doesn't end
up at one machine.

354
00:15:10,566 --> 00:15:10,786
>> Okay.

355
00:15:11,076 --> 00:15:13,866
>> So, the request can be
distributed over all machines.

356
00:15:14,286 --> 00:15:14,956
>> Okay, good.

357
00:15:14,956 --> 00:15:18,166
So now, if we get an inbound
HTTP request, we somehow want

358
00:15:18,166 --> 00:15:22,236
to distribute that request over
all of the various webservers

359
00:15:22,236 --> 00:15:25,586
that we might have whether
it's two webservers or 200.

360
00:15:25,586 --> 00:15:28,326
The problem really is still the
same, if it's more than one;

361
00:15:28,596 --> 00:15:30,356
we have to somehow
figure that out.

362
00:15:30,536 --> 00:15:33,386
So, let me put up a fairly
generic picture here -

363
00:15:33,386 --> 00:15:36,366
let me flip past those
to this guy here.

364
00:15:36,626 --> 00:15:39,646
So, if we have a whole bunch
of servers on the bottom here,

365
00:15:39,646 --> 00:15:41,366
server one, two, dot dot dot --

366
00:15:41,366 --> 00:15:43,356
and on the top, we have
some number of clients,

367
00:15:43,356 --> 00:15:44,766
just random people
on the Internet.

368
00:15:45,076 --> 00:15:46,516
We need to interpose now,

369
00:15:46,616 --> 00:15:48,646
some kind of black box
that's generally called a

370
00:15:48,646 --> 00:15:51,216
"load balancer" depicted
here as a rectangle

371
00:15:51,516 --> 00:15:53,976
so that the traffic
coming from people

372
00:15:53,976 --> 00:15:57,006
on the Internet is somehow
distributed or balanced

373
00:15:57,726 --> 00:16:01,316
across our various
"back-end" servers so to speak.

374
00:16:01,516 --> 00:16:03,976
So, it might still be the
case that server one and two

375
00:16:03,976 --> 00:16:08,076
and so forth, have unique
IP addresses but now,

376
00:16:08,076 --> 00:16:11,826
when a user types in something
dot com and hits enter,

377
00:16:12,846 --> 00:16:15,456
what IP address should
we return?

378
00:16:16,346 --> 00:16:19,676
How do we go about achieving
the equivalent of this man

379
00:16:19,676 --> 00:16:22,156
in the middle who can
somehow balance load

380
00:16:22,156 --> 00:16:23,426
across all end servers?

381
00:16:24,096 --> 00:16:28,996
>> Well, let - return the IP
address from the balancer?

382
00:16:28,996 --> 00:16:31,856
Send the request to
the load balancer

383
00:16:31,856 --> 00:16:36,456
and let the load balancer handle
which computer to send it to.

384
00:16:36,456 --> 00:16:37,026
>> Okay, good.

385
00:16:37,026 --> 00:16:40,706
So, instead of DNS, returning
the IP address of server one

386
00:16:40,706 --> 00:16:42,496
or server two or
server dot dot dot,

387
00:16:42,716 --> 00:16:44,986
you could instead return the
equivalent of the IP address

388
00:16:44,986 --> 00:16:47,286
of this black box,
the load balancer,

389
00:16:47,286 --> 00:16:49,956
and then let the load
balancer figure out how

390
00:16:49,956 --> 00:16:52,776
to actually route data to
those back-end servers.

391
00:16:52,776 --> 00:16:53,916
So, that's actually
pretty clean.

392
00:16:54,166 --> 00:16:56,496
So now, if the load balancer
has a public IP address,

393
00:16:56,896 --> 00:16:58,576
the back-end server's
now technically --

394
00:16:58,576 --> 00:17:00,606
you don't even need
public IP addresses.

395
00:17:00,606 --> 00:17:02,366
They can instead have
private addresses

396
00:17:02,846 --> 00:17:05,026
and what was the key
distinction between public

397
00:17:05,026 --> 00:17:07,926
and private IPs back
in Lecture Zero?

398
00:17:08,346 --> 00:17:12,036
Anyone -- anyone over here?

399
00:17:12,266 --> 00:17:12,686
Yeah? Louis?

400
00:17:12,806 --> 00:17:16,556
>> The rest of the world
can't see private IPs.

401
00:17:16,626 --> 00:17:18,446
>> Exactly, so the rest
of the world by definition

402
00:17:18,446 --> 00:17:20,746
of private can't see
private IP addresses.

403
00:17:21,346 --> 00:17:25,116
Then -- so that seems to have
some nice privacy properties

404
00:17:25,116 --> 00:17:28,176
and that if no one else can see
them, they can't address them.

405
00:17:28,176 --> 00:17:29,626
So those servers, just by nature

406
00:17:29,626 --> 00:17:32,856
of this privacy can't be
contacted at least directly

407
00:17:32,856 --> 00:17:35,506
by random adversaries,
bad guys, on the Internet.

408
00:17:35,566 --> 00:17:36,246
So, that's a plus.

409
00:17:36,606 --> 00:17:37,976
Moreover, the world
has been running

410
00:17:37,976 --> 00:17:40,606
out of version four
IP addresses,

411
00:17:40,656 --> 00:17:44,486
the 32-bit IP address is become
into scarcity for some time now

412
00:17:45,066 --> 00:17:47,296
so it's just hard or
sometimes expensive

413
00:17:47,296 --> 00:17:48,756
to get enough IP addresses

414
00:17:48,756 --> 00:17:51,096
to handle the various
servers that you might buy.

415
00:17:51,096 --> 00:17:52,366
So this alleviates
that pressure.

416
00:17:52,366 --> 00:17:55,676
Now, when we need one IP and
not multiple for our servers

417
00:17:55,676 --> 00:17:57,306
because we can give
these back-end servers

418
00:17:57,596 --> 00:18:00,976
like an odd number like
192.168 which most of you have

419
00:18:00,976 --> 00:18:03,366
in your home routers
probably or 10 dot something

420
00:18:03,676 --> 00:18:07,296
or 172.16 something, all
of those to mark the start

421
00:18:07,296 --> 00:18:08,606
of a private IP address.

422
00:18:08,836 --> 00:18:09,666
So, that works.

423
00:18:09,666 --> 00:18:12,706
All right, so the load balancer
has its own IP address now.

424
00:18:12,966 --> 00:18:14,966
It gets a request from some
client on the Internet.

425
00:18:15,586 --> 00:18:19,236
How, using jargon from
Lecture Zero onward,

426
00:18:19,426 --> 00:18:24,066
can the load balancer decide
or get that data to one

427
00:18:24,066 --> 00:18:25,226
of the back-end servers?

428
00:18:25,896 --> 00:18:27,466
How could we go about
implementing that?

429
00:18:28,156 --> 00:18:28,476
Axle?

430
00:18:28,896 --> 00:18:31,646
>> Well, probably first I'd
want to figure out which server

431
00:18:31,646 --> 00:18:33,906
to send it to, so
you'd want to check

432
00:18:34,216 --> 00:18:38,496
if somebody has available CPU
cycles that they're not using.

433
00:18:38,496 --> 00:18:38,816
>> Okay.

434
00:18:39,166 --> 00:18:41,466
>> So -- and once you see
that there's one server

435
00:18:41,466 --> 00:18:46,226
with enough CPU cycles
to handle that request,

436
00:18:46,226 --> 00:18:48,696
[inaudible] inside
your server network --

437
00:18:48,956 --> 00:18:49,146
>> Okay.

438
00:18:49,356 --> 00:18:52,446
>> -- to that machine to
get back whatever it is

439
00:18:52,526 --> 00:18:53,486
that the client requested

440
00:18:53,746 --> 00:18:55,636
and then the load balancer
sends it [inaudible].

441
00:18:55,636 --> 00:18:56,076
>> Excellent.

442
00:18:56,216 --> 00:18:58,796
So, this request arrives
then at the load balancer,

443
00:18:59,056 --> 00:19:01,156
the load balancer
decides to whom he wants

444
00:19:01,156 --> 00:19:03,996
to send this packet, server
one or two or dot dot dot,

445
00:19:04,096 --> 00:19:06,696
and you can make that decision
based on any number of factors.

446
00:19:06,696 --> 00:19:10,346
So, Axle proposed doing it based
on load like who is the busiest

447
00:19:10,346 --> 00:19:11,596
versus the least busy.

448
00:19:11,816 --> 00:19:15,136
Odds are I should send my
request to the least busy server

449
00:19:15,136 --> 00:19:18,636
in the interest of optimizing
performance all around.

450
00:19:18,926 --> 00:19:22,216
So, let's assume that
there's some way as demarcated

451
00:19:22,216 --> 00:19:23,876
by those black arrows,
like talking

452
00:19:23,876 --> 00:19:26,106
to those back-end servers and
saying, "Hey, how busy are you?

453
00:19:26,106 --> 00:19:26,666
Let me know."

454
00:19:26,986 --> 00:19:29,016
So, now the load balancer
figures out it wants

455
00:19:29,016 --> 00:19:30,926
to send this particular
request to server one,

456
00:19:31,196 --> 00:19:32,786
so it sends that request

457
00:19:32,786 --> 00:19:35,076
to server one using
similar mechanisms,

458
00:19:35,076 --> 00:19:37,656
TCP IP much like the
packet, how it traveled

459
00:19:37,656 --> 00:19:39,016
to the load balancer
in the first place.

460
00:19:39,406 --> 00:19:42,856
The server then gets the packet,
does its thing and says, "Oh,

461
00:19:42,856 --> 00:19:44,806
they want some HTML
file, here it is."

462
00:19:45,016 --> 00:19:46,746
The response goes to
the load balancer,

463
00:19:46,746 --> 00:19:49,956
the load balancer then responds
to the client, and voila.

464
00:19:50,456 --> 00:19:51,216
So, that works.

465
00:19:51,636 --> 00:19:54,386
What are some alternatives
to load balancing based

466
00:19:54,386 --> 00:19:59,206
on the actual load
on the server?

467
00:19:59,206 --> 00:20:01,976
So, load in general refers
to how busy a server is.

468
00:20:02,506 --> 00:20:04,346
So, what's an even
simpler approach than that,

469
00:20:04,516 --> 00:20:06,176
because that -- frankly,
that sounds a little complex?

470
00:20:06,176 --> 00:20:09,056
We've not talked at all about
how one device can query another

471
00:20:09,126 --> 00:20:11,816
for characteristics
like, "How busy are you?"

472
00:20:11,816 --> 00:20:12,746
Even though it's possible.

473
00:20:12,746 --> 00:20:12,986
Axle?

474
00:20:13,236 --> 00:20:16,166
>> Well, first let me point
out a downside with that,

475
00:20:16,296 --> 00:20:17,616
you [inaudible] every file --

476
00:20:18,496 --> 00:20:20,906
every file the website
has on every server.

477
00:20:20,906 --> 00:20:24,536
So, if you would instead say
have one server containing all

478
00:20:24,766 --> 00:20:28,066
the HTML and one running
and containing all the PHP,

479
00:20:28,066 --> 00:20:31,826
you would then see --
well, okay, bad example.

480
00:20:32,086 --> 00:20:33,626
One example of one server

481
00:20:33,626 --> 00:20:34,986
with all the images
[inaudible] HTML --

482
00:20:35,446 --> 00:20:35,966
>> Okay.

483
00:20:36,096 --> 00:20:37,716
>> -- [inaudible]
to request an image.

484
00:20:37,856 --> 00:20:39,596
It sends it to the
image server --

485
00:20:39,596 --> 00:20:39,826
>> Okay.

486
00:20:39,976 --> 00:20:42,536
>> -- and if it's an HTML
request, to the HTML server.

487
00:20:42,626 --> 00:20:43,276
>> Okay, good.

488
00:20:43,276 --> 00:20:45,796
So, the implication of the
previous story that Axle told is

489
00:20:45,796 --> 00:20:48,256
that under this model, server
one, two, and so forth,

490
00:20:48,366 --> 00:20:52,456
all need to be identical, have
the same content which is nice

491
00:20:52,456 --> 00:20:54,626
and then it doesn't matter
to whom you send the request.

492
00:20:54,626 --> 00:20:58,346
The downside is now, you're
using N times as much disc space

493
00:20:58,586 --> 00:21:00,036
as you might otherwise need to,

494
00:21:00,036 --> 00:21:03,426
but that's perhaps the price you
pay for having this redundancy

495
00:21:03,426 --> 00:21:05,966
or to having this
horizontal scalability.

496
00:21:06,186 --> 00:21:08,306
Or instead, you could
have dedicated servers --

497
00:21:08,306 --> 00:21:10,076
these are for HTML,
these are for gifs,

498
00:21:10,076 --> 00:21:12,746
these are for movie files and
the like, and you could do

499
00:21:12,746 --> 00:21:16,006
that just by having different
URLs, different host names.

500
00:21:16,006 --> 00:21:19,506
This is, for instance,
"Images dot something dot com."

501
00:21:19,506 --> 00:21:21,326
"This is videos dot
something dot com."

502
00:21:21,326 --> 00:21:22,666
And then the load
balancer could take

503
00:21:22,666 --> 00:21:26,196
into account the host
HTTP header to decide

504
00:21:26,246 --> 00:21:27,876
which direction it should go in.

505
00:21:27,876 --> 00:21:29,116
So, that could work for us.

506
00:21:29,366 --> 00:21:31,996
All right, so what's an
even simpler puristic

507
00:21:31,996 --> 00:21:35,126
than asking a back-end server,
"How busy are you right now?"

508
00:21:35,416 --> 00:21:37,046
Like, if you have no
idea how to do that?

509
00:21:37,456 --> 00:21:39,496
How instead, could
we balance load

510
00:21:39,496 --> 00:21:43,256
across an arbitrary
number of servers?

511
00:21:43,406 --> 00:21:45,146
Think again back
to Lecture Zero.

512
00:21:45,866 --> 00:21:49,906
You can do all of this with only
Lecture Zero under our belt.

513
00:21:53,756 --> 00:21:55,256
So, let's quickly
tell the story.

514
00:21:55,256 --> 00:21:57,226
I type in something
dot com into a browser;

515
00:21:57,456 --> 00:21:58,716
I hit "Enter," what happens?

516
00:21:58,716 --> 00:21:58,806
Jack?

517
00:21:58,806 --> 00:22:03,516
>> We create a packet to send.

518
00:22:03,516 --> 00:22:06,326
>> Okay, packet to send
-- to whom do we send it?

519
00:22:06,626 --> 00:22:09,696
>> We send it to someplace that
will determine the IP address

520
00:22:09,696 --> 00:22:11,916
of where we're sending it to.

521
00:22:12,106 --> 00:22:13,786
>> Okay, good -- something that
will determine the IP address

522
00:22:13,786 --> 00:22:15,306
of where we're sending it to
-- what's that thing called?

523
00:22:15,706 --> 00:22:17,796
Isaac, what's that called?

524
00:22:17,796 --> 00:22:18,306
>> A router?

525
00:22:18,556 --> 00:22:22,006
>> Not router -- the
routers get involved but --

526
00:22:22,806 --> 00:22:23,746
>> The DNS.

527
00:22:23,746 --> 00:22:26,646
>> The DNS server, domain name
system server, so that server

528
00:22:26,646 --> 00:22:29,786
in the world, whole bunches
of them that whose purpose

529
00:22:29,786 --> 00:22:32,976
in life is to translate host
names to IPs and vice versa.

530
00:22:33,176 --> 00:22:35,716
So, I'm going to
pause the story there.

531
00:22:36,066 --> 00:22:41,066
That seems to be an opportunity
now for us to return something.

532
00:22:41,066 --> 00:22:41,166
Yeah?

533
00:22:41,466 --> 00:22:44,346
>> Could you do some DNS tricks

534
00:22:44,636 --> 00:22:46,936
and return a different
IP address based

535
00:22:46,936 --> 00:22:48,656
on what the user requested?

536
00:22:48,836 --> 00:22:50,886
>> Good, so this black
box, this load balancer,

537
00:22:50,886 --> 00:22:53,406
maybe it's just a
fancy DNS setup.

538
00:22:53,536 --> 00:22:55,626
Whereby instead of
returning the IP address

539
00:22:55,626 --> 00:22:57,536
of the load balancer itself,

540
00:22:57,536 --> 00:23:01,006
maybe instead the DNS server
just returns the IP address

541
00:23:01,006 --> 00:23:03,246
of server one the
first time someone asks

542
00:23:03,246 --> 00:23:04,236
for something dot com.

543
00:23:04,456 --> 00:23:06,546
And the next time someone
requests something dot com,

544
00:23:06,546 --> 00:23:08,866
it returns the IP address
of server two followed

545
00:23:08,866 --> 00:23:11,536
by server three followed by
dot dot dot and the wrapping

546
00:23:11,536 --> 00:23:13,926
around eventually
to server one again.

547
00:23:13,926 --> 00:23:16,166
So, this is actually
generally called "Round Robin"

548
00:23:16,376 --> 00:23:18,046
and you can do this
fairly easily.

549
00:23:18,046 --> 00:23:21,456
This is just a snippet of a
popular DNS server called BIND,

550
00:23:21,826 --> 00:23:26,926
Berkeley Internet Name
Demon, I believe is the D.

551
00:23:27,166 --> 00:23:28,976
And this just suggests
that if you want

552
00:23:28,976 --> 00:23:31,596
to have multiple IP
addresses for host name called

553
00:23:31,596 --> 00:23:35,616
"dub dub dub" you mention
"A" which denotes A record

554
00:23:35,616 --> 00:23:37,386
and just refers to
an end pointer here.

555
00:23:37,386 --> 00:23:39,906
But A is the same
as in Lecture Zero

556
00:23:40,056 --> 00:23:43,366
and then you just enumerate the
IP address one after the other

557
00:23:43,676 --> 00:23:47,316
and by default, this particular
DNS server, very popular one,

558
00:23:48,116 --> 00:23:53,006
BIND, will return a different
IP address for each request.

559
00:23:53,686 --> 00:23:54,316
So, that's nice.

560
00:23:54,416 --> 00:23:55,026
It's simple.

561
00:23:55,026 --> 00:23:56,466
Again, it uses only
some knowledge

562
00:23:56,466 --> 00:23:58,596
from Lecture Zero even though
granted, you have to know how

563
00:23:58,596 --> 00:23:59,956
to configure the DNS server,

564
00:24:00,116 --> 00:24:02,786
but you don't need any fancy
bi-directional communication

565
00:24:02,786 --> 00:24:04,296
with the back-end
servers in this model.

566
00:24:04,296 --> 00:24:04,876
So, that's nice.

567
00:24:05,676 --> 00:24:08,976
But there's a price we
pay for the simplicity.

568
00:24:08,976 --> 00:24:12,246
If we only do Round Robin,
where again, we just spit

569
00:24:12,246 --> 00:24:14,416
out a different IP
address each time

570
00:24:14,886 --> 00:24:16,686
and let me make this
more concrete just

571
00:24:16,686 --> 00:24:18,676
so this isn't quite as abstract.

572
00:24:18,676 --> 00:24:22,666
Let me open up a terminal
program here and do NS Lookup

573
00:24:22,666 --> 00:24:24,766
for name server lookup
of Google.com.

574
00:24:25,036 --> 00:24:28,266
This is exactly what Google
does, at least in part.

575
00:24:28,446 --> 00:24:31,836
Their load balancing
solution is more sophisticated

576
00:24:31,836 --> 00:24:33,136
than this list suggests,

577
00:24:33,206 --> 00:24:36,946
but indeed Google's DNS server
returns multiple IP addresses

578
00:24:36,946 --> 00:24:37,576
each time.

579
00:24:37,956 --> 00:24:46,226
So, if this is so simple to
implement, what's the catch?

580
00:24:46,396 --> 00:24:46,656
Axle?

581
00:24:46,656 --> 00:24:49,046
>> It's not a very smart
solution because just --

582
00:24:49,916 --> 00:24:54,516
there could be -- the case could
be that one server gets all

583
00:24:54,516 --> 00:24:56,686
of the really tough
and hard requests --

584
00:24:56,686 --> 00:24:56,906
>> Good.

585
00:24:57,096 --> 00:24:58,836
>> -- that take a lot
of processing power.

586
00:24:59,066 --> 00:25:01,406
The other ones just
get the [inaudible].

587
00:25:01,856 --> 00:25:02,846
There's no way to know that.

588
00:25:02,966 --> 00:25:04,786
>> Good, so just by bad luck,

589
00:25:04,786 --> 00:25:07,146
one of the servers could
just get a really --

590
00:25:07,266 --> 00:25:08,686
a real power user, someone

591
00:25:08,686 --> 00:25:11,296
who is really doing something
computationally difficult.

592
00:25:11,716 --> 00:25:15,446
Like, I don't know,
what's a good --

593
00:25:15,576 --> 00:25:18,566
sending lots of mail or someone
else is just kind of logging in

594
00:25:18,566 --> 00:25:21,166
and poking around at a much
slower rate and we could come

595
00:25:21,166 --> 00:25:23,416
up with even more sophisticated
examples than that.

596
00:25:23,416 --> 00:25:26,416
But over time, server
one might just happen

597
00:25:26,416 --> 00:25:30,436
to get more heavy weight
users than other servers.

598
00:25:30,436 --> 00:25:31,466
So, what's the implication?

599
00:25:31,676 --> 00:25:34,726
Well, Round Robin is still
going to keep sending more

600
00:25:34,726 --> 00:25:38,656
and more users to that server
one-Nth of the time just

601
00:25:38,656 --> 00:25:39,806
by nature of Round Robin.

602
00:25:40,556 --> 00:25:42,106
So, that's not so good.

603
00:25:43,566 --> 00:25:50,746
What else causes problems here
or what else breaks potentially?

604
00:25:50,746 --> 00:25:57,716
So, back to the Lecture Zero
story, I type something dot com,

605
00:25:57,716 --> 00:26:00,606
I hit "Enter", my browser
sends a request to the DNS --

606
00:26:00,966 --> 00:26:03,596
or my operating systems sends
a request to the DNS server,

607
00:26:03,596 --> 00:26:05,466
it gets the IP address
and in this model, it's --

608
00:26:05,806 --> 00:26:10,086
the IP address of one of these
servers, then I send my packet

609
00:26:10,086 --> 00:26:12,716
as Jack proposed to
that particular server,

610
00:26:12,716 --> 00:26:14,706
get back a response, story ends.

611
00:26:15,246 --> 00:26:17,886
But then a few seconds
later, I visit another link

612
00:26:17,886 --> 00:26:19,696
on something dot
com and hit "Enter",

613
00:26:20,006 --> 00:26:21,726
which part of the
story now changes?

614
00:26:23,056 --> 00:26:23,986
Jack?

615
00:26:23,986 --> 00:26:26,876
>> Is it now going to send to
the same server [inaudible]

616
00:26:26,876 --> 00:26:30,086
or does it send it
to a new server?

617
00:26:30,086 --> 00:26:30,846
>> Good question.

618
00:26:32,416 --> 00:26:33,816
>> Exactly where
it's coming from?

619
00:26:34,386 --> 00:26:37,026
>> So, how does the story change
and that'll give us the answer?

620
00:26:37,026 --> 00:26:37,396
Axle?

621
00:26:37,926 --> 00:26:39,146
>> Well, it has to send it

622
00:26:41,216 --> 00:26:43,166
to a new server otherwise
Round Robin wouldn't work.

623
00:26:43,166 --> 00:26:43,686
>> Oh, ideally, yes.

624
00:26:43,686 --> 00:26:46,006
If you want a truly
uniform distribution

625
00:26:46,006 --> 00:26:47,106
across all end servers,
then the DNS server has

626
00:26:47,106 --> 00:26:48,636
to return another
response and I'd argue,

627
00:26:48,896 --> 00:26:52,136
the DNS server will return a
different response the next time

628
00:26:52,136 --> 00:26:52,816
it is queried.

629
00:26:53,476 --> 00:26:53,916
But --

630
00:26:54,576 --> 00:26:55,976
>> Oh, it's not queried.

631
00:26:56,296 --> 00:26:56,566
>> Why?

632
00:26:56,656 --> 00:26:57,646
>> Because it's saved.

633
00:26:58,306 --> 00:26:59,246
>> Saved or --

634
00:26:59,246 --> 00:27:03,116
>> No, but it's -- the --
the IP address is stored --

635
00:27:03,806 --> 00:27:04,046
>> Where?

636
00:27:04,236 --> 00:27:06,596
>> -- [inaudible] open session
otherwise it's really useless

637
00:27:06,706 --> 00:27:09,696
to query the DNS server for the
same thing over and over again.

638
00:27:09,696 --> 00:27:12,146
>> Good. So, recall these
caches back in Lecture Zero.

639
00:27:12,146 --> 00:27:14,786
We talked about the
implications of the good parts

640
00:27:14,786 --> 00:27:17,866
of caching whereby as Axle's
proposing, there's no reason

641
00:27:17,866 --> 00:27:21,486
for Chrome or IE to send the
same DNS request every single

642
00:27:21,486 --> 00:27:23,926
time you click a link
on something dot com.

643
00:27:23,926 --> 00:27:25,696
That would just be
a waste of time.

644
00:27:25,696 --> 00:27:27,746
You're going to lose some number
of milliseconds every time

645
00:27:27,746 --> 00:27:29,696
that happens or worse
yet, a second or two.

646
00:27:29,986 --> 00:27:32,456
So, instead your operating
system typically caches

647
00:27:32,506 --> 00:27:33,316
these responses.

648
00:27:33,586 --> 00:27:36,666
Your browser typically caches
these responses as well,

649
00:27:36,856 --> 00:27:39,356
and so you just don't
need to do those lookups.

650
00:27:39,456 --> 00:27:42,206
So, if you do happen to be that
power user that's doing a heck

651
00:27:42,206 --> 00:27:45,176
of a lot of work of
whatever sort on server one,

652
00:27:46,156 --> 00:27:48,696
the next guy is going to
be sent to server two,

653
00:27:48,696 --> 00:27:50,846
not you with your
subsequent request.

654
00:27:51,326 --> 00:27:54,436
So, caching too can contribute
to a disproportionate amount

655
00:27:54,436 --> 00:27:58,606
of load on certain servers
largely due to bad luck.

656
00:27:58,746 --> 00:28:00,956
Indeed in DNS, we
didn't spend much time

657
00:28:00,956 --> 00:28:01,996
on this particular detail

658
00:28:01,996 --> 00:28:04,636
but there's typically
expiration times, TTLs,

659
00:28:04,766 --> 00:28:09,046
Time to Live values associated
with an answer from a DNS server

660
00:28:09,226 --> 00:28:12,576
and that's typically an hour,
or five minutes, or a day.

661
00:28:12,576 --> 00:28:13,356
It totally depends

662
00:28:13,356 --> 00:28:15,656
on who controls the DNS
server, what the value is.

663
00:28:15,886 --> 00:28:19,116
But that suggests too, that
if you are this power user

664
00:28:19,116 --> 00:28:22,856
on server one, it might be a few
minutes, or hours, or even days,

665
00:28:22,986 --> 00:28:26,496
until you get assigned to
some other server simply

666
00:28:26,496 --> 00:28:29,366
because your TTL
has expired by then.

667
00:28:29,806 --> 00:28:30,926
So, it's nice and simple.

668
00:28:30,926 --> 00:28:32,846
We can do it with a
simple configuration change

669
00:28:33,056 --> 00:28:35,426
but it doesn't necessarily
solve all of our problems.

670
00:28:35,426 --> 00:28:38,706
So, in fact, the approach Axle
proposed first is actually

671
00:28:38,706 --> 00:28:42,706
pretty good whereby you don't
use DNS space to Round Robin.

672
00:28:43,016 --> 00:28:45,116
Rather, more sophisticated
approach would be

673
00:28:45,116 --> 00:28:48,386
to let the load balancer to
decide to whom to send you

674
00:28:48,386 --> 00:28:50,006
in the back-end and the
load balancer can make

675
00:28:50,006 --> 00:28:52,336
that decision using any
number of puristics.

676
00:28:52,336 --> 00:28:55,646
It could even use Round Robin
or randomness because at

677
00:28:55,646 --> 00:28:58,036
that point, you don't have
to worry about caching issues

678
00:28:58,286 --> 00:29:00,806
because the DNS server
has only returned one IP.

679
00:29:01,036 --> 00:29:03,386
So -- but that still
leads you to the risk

680
00:29:03,386 --> 00:29:05,786
that you'll be sent -- putting
too much load on some server

681
00:29:06,036 --> 00:29:07,376
so we could take, for instance,

682
00:29:07,376 --> 00:29:09,386
server load into
account at that point.

683
00:29:09,736 --> 00:29:11,046
But there is something
else that breaks.

684
00:29:11,046 --> 00:29:13,566
If we fast forward mid-semester
to when we started talking

685
00:29:14,246 --> 00:29:19,906
about cookies and HTTP
and sessions and PHP.

686
00:29:20,456 --> 00:29:22,846
To spark discussion, I propose

687
00:29:22,846 --> 00:29:25,336
that sessions have
just broken in PHP.

688
00:29:25,336 --> 00:29:30,046
If our back-end servers
are PHP-based websites

689
00:29:30,806 --> 00:29:34,946
and they are using the
session super global,

690
00:29:35,126 --> 00:29:37,596
load balancing seems
to break this model.

691
00:29:37,596 --> 00:29:38,326
Why, Jack?

692
00:29:38,426 --> 00:29:42,306
>> Because now, the different
servers have different people

693
00:29:42,406 --> 00:29:46,236
sessions so although one might
have my session and by then,

694
00:29:46,586 --> 00:29:49,386
I'm redirecting from
server one to server two,

695
00:29:49,386 --> 00:29:50,866
server two might
not have my session.

696
00:29:50,866 --> 00:29:51,466
>> Exactly.

697
00:29:51,466 --> 00:29:52,566
>> Therefore, everything's lost.

698
00:29:52,856 --> 00:29:54,346
>> So, sessions recall, tend

699
00:29:54,346 --> 00:29:55,776
to be specific to
the given machine.

700
00:29:55,776 --> 00:29:57,836
We saw examples involving
slash temp

701
00:29:57,836 --> 00:29:59,736
which is a temporary
directory on the Linux system

702
00:30:00,046 --> 00:30:03,106
where sessions were
typically saved as text files,

703
00:30:03,196 --> 00:30:03,976
serialized text files.

704
00:30:04,736 --> 00:30:07,366
So, that means though, that
your session might be sitting

705
00:30:07,366 --> 00:30:09,446
on the hard drive of
server one and yet if

706
00:30:09,446 --> 00:30:12,766
by random chance you are sent
by a Round Robin to server two

707
00:30:12,766 --> 00:30:16,276
or server three instead of
server one, in the worst case,

708
00:30:16,276 --> 00:30:18,056
you're going to see the same
website but you're going

709
00:30:18,056 --> 00:30:19,866
to be told to log in
again for instance,

710
00:30:19,866 --> 00:30:22,166
because that server doesn't
know that you've logged in.

711
00:30:22,406 --> 00:30:25,176
Okay, find -- you kind of
bite your tongue and you type

712
00:30:25,176 --> 00:30:27,386
in your user name and
password again and hit "Enter"

713
00:30:27,596 --> 00:30:29,636
and suppose you're
a really good sport

714
00:30:29,636 --> 00:30:31,116
and you do this for
all end servers.

715
00:30:31,116 --> 00:30:33,706
You have no idea why something
dot com keeps prompting you

716
00:30:33,706 --> 00:30:36,446
to log in but eventually, you
will have a session cookie

717
00:30:36,576 --> 00:30:37,926
on all of those servers.

718
00:30:37,926 --> 00:30:40,486
The catch then though is that
if something dot com is an

719
00:30:40,486 --> 00:30:42,246
E-Commerce site and
you're adding things

720
00:30:42,246 --> 00:30:44,876
to your shopping cart, now
you literally have put a book

721
00:30:44,876 --> 00:30:47,376
in your cart over here, a
different book in this cart,

722
00:30:47,376 --> 00:30:49,626
a different book in this
cart and when you check out,

723
00:30:49,626 --> 00:30:51,246
you can't check out
the aggregate.

724
00:30:52,056 --> 00:30:54,316
So, this is a very
non-trivial problem now.

725
00:30:54,806 --> 00:30:55,106
Axle?

726
00:30:55,396 --> 00:30:57,426
>> But this wouldn't
happen if you --

727
00:30:57,826 --> 00:31:01,166
if you had dedicated machines
[inaudible] one machine running

728
00:31:01,166 --> 00:31:04,676
PHP and then one machine
serving all the images.

729
00:31:04,766 --> 00:31:05,366
>> Very true.

730
00:31:05,366 --> 00:31:07,786
So, if we have horizontally
scaled in the sense

731
00:31:07,786 --> 00:31:09,766
that we've factored
out disparate services,

732
00:31:09,766 --> 00:31:13,016
this is our PHP server,
this is our gif server,

733
00:31:13,016 --> 00:31:14,296
this is our video server.

734
00:31:14,496 --> 00:31:16,486
Then indeed, this
problem would not arise

735
00:31:16,486 --> 00:31:19,796
because presumably all the
PHP traffic would get routed

736
00:31:19,796 --> 00:31:20,696
to the PHP server.

737
00:31:20,696 --> 00:31:23,846
But an obvious push-back
to that solution is what?

738
00:31:25,166 --> 00:31:25,446
Isaac?

739
00:31:25,686 --> 00:31:31,146
>> Well, if one of them crashes
and you lose all images or --

740
00:31:31,186 --> 00:31:31,456
>> Okay, good.

741
00:31:31,456 --> 00:31:33,146
There's no redundancy
which is not good

742
00:31:33,146 --> 00:31:34,516
for uptime if anything breaks.

743
00:31:34,676 --> 00:31:35,166
Axle?

744
00:31:35,346 --> 00:31:38,826
>> And also, well at
some point in time,

745
00:31:38,916 --> 00:31:42,576
if you get popular enough,
that one PHP server's not going

746
00:31:42,576 --> 00:31:43,406
to be able to handle
everything --

747
00:31:43,406 --> 00:31:43,496
>> Good.

748
00:31:43,496 --> 00:31:44,456
>> -- it need to [inaudible].

749
00:31:44,616 --> 00:31:45,646
>> The end of story's the same.

750
00:31:45,646 --> 00:31:47,836
As soon as you get popular,
you get too much load

751
00:31:47,836 --> 00:31:49,526
for a single PHP
server, then we have

752
00:31:49,526 --> 00:31:51,096
to solve this problem anyway.

753
00:31:51,456 --> 00:31:54,616
So, how do we go about
solving this problem?

754
00:31:54,616 --> 00:31:58,716
This seems to be a real pain,
this one and to be clear;

755
00:31:58,716 --> 00:32:00,556
the problem now is that inasmuch

756
00:32:00,556 --> 00:32:03,446
as sessions are typically
implemented per server

757
00:32:03,706 --> 00:32:06,376
in the form of like a text
file like we saw in slash temp,

758
00:32:06,886 --> 00:32:10,876
then you can't really
use Round Robin.

759
00:32:10,876 --> 00:32:13,916
You can't really use load
-- true load balancing,

760
00:32:13,916 --> 00:32:15,956
taking into account each
server's load because you need

761
00:32:15,956 --> 00:32:18,956
to make sure that Alice,
if she's initially sent

762
00:32:18,956 --> 00:32:21,966
to server one, subsequently
gets sent to server one again,

763
00:32:21,966 --> 00:32:24,896
and again, and again, for
at least, you know, an hour

764
00:32:24,896 --> 00:32:27,896
or a day or some amount of time
so that her session is useful.

765
00:32:27,936 --> 00:32:28,186
Jack?

766
00:32:28,606 --> 00:32:31,536
>> Is there a way to create
a server that's especially

767
00:32:31,596 --> 00:32:33,796
dedicated to store
everyone's session

768
00:32:33,796 --> 00:32:36,246
and all the sessions
are started there and --

769
00:32:36,246 --> 00:32:36,666
>> Excellent.

770
00:32:37,326 --> 00:32:38,576
Yes, so absolutely.

771
00:32:38,576 --> 00:32:42,136
We could just continue this
idea of factorization and factor

772
00:32:42,136 --> 00:32:44,246
out not the various
types of files,

773
00:32:44,466 --> 00:32:46,476
but a service like
session state.

774
00:32:46,826 --> 00:32:49,126
So, if we instead had a
file server, you know,

775
00:32:49,126 --> 00:32:52,946
like a big external hard drive,
so to speak, that is connected

776
00:32:52,946 --> 00:32:55,396
to all of the servers,
one and two and three,

777
00:32:55,626 --> 00:32:57,916
so that anytime they
store session data,

778
00:32:58,456 --> 00:33:01,496
they store it there instead
of on their own hard drive.

779
00:33:01,496 --> 00:33:03,226
Then this way, we
could share state.

780
00:33:03,666 --> 00:33:05,646
So indeed, that could
be a solution here.

781
00:33:05,646 --> 00:33:06,066
Axle?

782
00:33:06,746 --> 00:33:11,006
>> What -- I don't know if the
load balancer has that function,

783
00:33:11,126 --> 00:33:12,966
but [inaudible] instead
of having an extra server,

784
00:33:13,076 --> 00:33:16,526
that all the other servers
can query the load balancer

785
00:33:16,526 --> 00:33:18,186
because all traffic goes
through that anyhow.

786
00:33:18,486 --> 00:33:20,756
What if the load balancer
stores the session?

787
00:33:20,756 --> 00:33:21,976
>> Okay, so that's
not bad at all.

788
00:33:21,976 --> 00:33:23,986
So, we already have a
man in the middle here.

789
00:33:23,986 --> 00:33:26,166
It's a black box but there's no
reason it couldn't be a server

790
00:33:26,166 --> 00:33:27,256
with hard disc space.

791
00:33:27,576 --> 00:33:29,556
So, why not put the sessions
on the load balancer?

792
00:33:29,556 --> 00:33:30,656
That could absolutely work.

793
00:33:31,076 --> 00:33:34,246
So, let me difficult then and
whether we put the sessions

794
00:33:34,246 --> 00:33:35,836
on the load balancer
or whatever --

795
00:33:35,836 --> 00:33:38,356
that's no longer a load balancer
then, it's obviously doing more.

796
00:33:38,356 --> 00:33:40,046
It's more of a server
that happens

797
00:33:40,046 --> 00:33:41,896
to be balancing load
and storing sessions.

798
00:33:42,306 --> 00:33:45,526
Whether we put sessions there
in that black box or elsewhere

799
00:33:45,526 --> 00:33:46,936
in a new box on the screen,

800
00:33:47,576 --> 00:33:51,106
we seem to have introduced
a weakness now

801
00:33:51,106 --> 00:33:56,746
in our network topology because
what if that machine breaks?

802
00:33:57,716 --> 00:33:58,946
It would seem to be the case

803
00:33:58,946 --> 00:34:01,366
that even though we have
N servers which in theory,

804
00:34:01,366 --> 00:34:03,436
those guys are never all
going to die at once,

805
00:34:03,496 --> 00:34:05,556
assuming that it's not
the power, electricity,

806
00:34:05,556 --> 00:34:08,226
or something stupid like
that that's somehow related

807
00:34:08,226 --> 00:34:08,866
to all of them.

808
00:34:09,166 --> 00:34:11,206
But odds are they're
not all just going to up

809
00:34:11,206 --> 00:34:13,756
and die simultaneously so we
have really good redundancy

810
00:34:13,756 --> 00:34:15,186
in our server model right now.

811
00:34:15,436 --> 00:34:18,126
But as soon as we introduce
just a database or file server

812
00:34:18,186 --> 00:34:22,596
for our sessions, if that guy
dies, then what was the point

813
00:34:22,596 --> 00:34:24,786
of spending all this money on
all these back-end servers?

814
00:34:25,166 --> 00:34:27,896
Our whole site goes down because
we have no ability to remember

815
00:34:27,896 --> 00:34:29,526
that people are logged in
if we can't remember they're

816
00:34:29,526 --> 00:34:29,896
logged in.

817
00:34:29,896 --> 00:34:30,716
No one can buy anything.

818
00:34:31,616 --> 00:34:32,936
So, how do we fix that problem?

819
00:34:33,156 --> 00:34:36,136
So, we've solved one problem
but if you think of that sort

820
00:34:36,136 --> 00:34:39,316
of old visual where you have
like a garden hose with lots

821
00:34:39,316 --> 00:34:41,996
of leaks in it and you plug
one of them with one hand,

822
00:34:41,996 --> 00:34:44,116
all of a sudden a new
leak springs up elsewhere.

823
00:34:44,276 --> 00:34:45,576
That's kind of what's
happened here.

824
00:34:45,726 --> 00:34:48,986
We solved the problem
of shared state but now,

825
00:34:48,986 --> 00:34:53,366
we've sacrificed some
robustness, some redundancy.

826
00:34:54,236 --> 00:34:54,976
How do we now fix the latter?

827
00:35:03,196 --> 00:35:03,556
Axle?

828
00:35:03,556 --> 00:35:07,156
>> It's probably not the
solution you're looking for,

829
00:35:07,156 --> 00:35:10,696
that the hardware on your little
data center could be built

830
00:35:11,056 --> 00:35:13,256
such that things like the
hard drive [inaudible] missing

831
00:35:13,546 --> 00:35:14,166
RAID type.

832
00:35:15,036 --> 00:35:18,926
>> Okay good so we could just
use sort of different approach

833
00:35:18,926 --> 00:35:21,756
to storing our data and
rather than just store it

834
00:35:21,756 --> 00:35:24,236
on the hard disk as usual, we
could use something called RAID.

835
00:35:24,296 --> 00:35:25,866
So actually this
is good way to tie

836
00:35:25,866 --> 00:35:28,416
in the thing we skipped
a moment ago.

837
00:35:28,416 --> 00:35:36,476
Let me just pull up
something to write on here.

838
00:35:36,676 --> 00:35:39,496
So Redundant Array

839
00:35:39,496 --> 00:35:42,736
of Independent Disks is a
technology more succinctly known

840
00:35:42,736 --> 00:35:43,526
as RAID.

841
00:35:43,706 --> 00:35:44,846
RAID can actually be used

842
00:35:44,846 --> 00:35:48,026
in desktop computers these
days even though it's not all

843
00:35:48,026 --> 00:35:48,556
that common.

844
00:35:48,606 --> 00:35:52,216
Some companies like Dell and
Apple make it relatively easy

845
00:35:52,216 --> 00:35:55,006
to use RAID on your system,
and what does this mean?

846
00:35:55,006 --> 00:35:56,876
Well RAID can come in
a few different forms.

847
00:35:56,916 --> 00:35:58,586
There's something
called RAID zero,

848
00:35:58,836 --> 00:36:02,236
there's something called
RAID1, there's something RAID5,

849
00:36:02,316 --> 00:36:04,836
there's something called RAID6,
there's something called RAID10

850
00:36:04,836 --> 00:36:06,896
and there's even more,
but there are some

851
00:36:06,896 --> 00:36:08,376
of the simplest ones
to talk about.

852
00:36:08,966 --> 00:36:13,426
So all of these technog -- all
of these variance of RAID assume

853
00:36:13,426 --> 00:36:15,676
that you have multiple hard
drives in your computer

854
00:36:15,676 --> 00:36:17,166
for different purposes
potentially.

855
00:36:17,466 --> 00:36:19,026
So in the world of RAID1 --

856
00:36:19,096 --> 00:36:22,396
RAID zero you typically
have two hard drives

857
00:36:22,716 --> 00:36:23,976
that are of identical size.

858
00:36:24,236 --> 00:36:27,646
Terabyte, two terabytes, 512
gigabytes whatever it is,

859
00:36:27,926 --> 00:36:29,776
two identical hard drives

860
00:36:29,776 --> 00:36:33,536
and you do what's called
stripe data across them.

861
00:36:33,886 --> 00:36:35,986
Whereby every time the
Operating System wants

862
00:36:35,986 --> 00:36:39,146
to save a file especially big
files, it will first write

863
00:36:39,146 --> 00:36:41,256
to this drive a bit
then to this one,

864
00:36:41,326 --> 00:36:42,926
then to this one,
then to this one.

865
00:36:42,926 --> 00:36:45,696
The motivation being these
hard drives are typically large

866
00:36:45,696 --> 00:36:47,906
and mechanical with
spinning platters

867
00:36:47,906 --> 00:36:49,106
like we discussed earlier,

868
00:36:49,346 --> 00:36:52,296
and so it might take this
guy a little while to write

869
00:36:52,296 --> 00:36:53,626
out some number of bits.

870
00:36:53,626 --> 00:36:55,466
Now it's going to be
split second in reality

871
00:36:55,706 --> 00:36:57,256
but that's a split second
we don't really have

872
00:36:57,256 --> 00:36:58,866
when we're trying to
service lots of users.

873
00:36:59,066 --> 00:37:02,056
So striping allows me to write
some data here then here,

874
00:37:02,056 --> 00:37:05,226
then here then here, then here
then here effectively doubling

875
00:37:05,556 --> 00:37:07,996
the speed at which I can write
files especially large ones

876
00:37:07,996 --> 00:37:08,476
to disks.

877
00:37:08,476 --> 00:37:10,766
So RAID zero is nice
for performance.

878
00:37:11,166 --> 00:37:13,356
However, RAID1 gives you
a very different property.

879
00:37:13,356 --> 00:37:17,716
With RAID1 you still have two
hard drives but you mirror data

880
00:37:17,956 --> 00:37:20,316
so to speak across them,
so that anytime you write

881
00:37:20,316 --> 00:37:24,036
out a file you store it
both places simultaneously.

882
00:37:24,256 --> 00:37:26,386
There's a bit of performance
overhead to writing it

883
00:37:26,386 --> 00:37:29,676
in two places albeit in
parallel, but the upside now is

884
00:37:29,676 --> 00:37:31,966
that either of these
drives can die

885
00:37:32,386 --> 00:37:34,426
and your data is still
perfectly intact,

886
00:37:34,586 --> 00:37:36,576
and it's actually an
amazing technology

887
00:37:36,576 --> 00:37:38,146
because if even you
just have this

888
00:37:38,146 --> 00:37:40,316
in your desktop computer,
you have two drives.

889
00:37:40,316 --> 00:37:43,256
One of them dies just because
of bad luck, there's a defect

890
00:37:43,256 --> 00:37:45,716
or its multiple years old
and it just up and died,

891
00:37:46,056 --> 00:37:47,856
so long as the other
one is still working.

892
00:37:47,856 --> 00:37:50,796
The theory behind RAID is that
you can then run to the store,

893
00:37:51,076 --> 00:37:53,836
buy another hard drive that's
at the same size or bigger,

894
00:37:54,066 --> 00:37:56,776
plug it into your
computer, boot back up,

895
00:37:56,776 --> 00:37:59,736
and most typically
automatically,

896
00:38:00,046 --> 00:38:03,856
the RAID array will
rebuild itself whereby all

897
00:38:03,856 --> 00:38:04,446
of the data that's

898
00:38:04,446 --> 00:38:07,326
on the remaining drive will
copy itself automatically

899
00:38:07,326 --> 00:38:09,206
over to the new one
and after a few minutes

900
00:38:09,206 --> 00:38:11,766
or hours you're back
to a safer place.

901
00:38:12,156 --> 00:38:14,306
Whereby now even the other
drive can up and die.

902
00:38:14,306 --> 00:38:16,696
Sometimes you have to run a
command or choose a menu option

903
00:38:16,696 --> 00:38:18,606
to induce that but
typically it's automatic.

904
00:38:18,896 --> 00:38:20,196
You can do it even sometimes

905
00:38:20,196 --> 00:38:22,206
in some machines while
the computer is still

906
00:38:22,206 --> 00:38:23,096
on so you don't even have

907
00:38:23,096 --> 00:38:25,866
to suffer any downtime,
so that's great.

908
00:38:25,866 --> 00:38:27,956
RAID10 is essentially the
combination of those two.

909
00:38:27,956 --> 00:38:30,756
You typically use four drives
and you have both striping

910
00:38:30,756 --> 00:38:33,656
and redundancy so you sort of
get the best of both worlds,

911
00:38:33,656 --> 00:38:35,686
but costs you twice as
much cause you need twice

912
00:38:35,686 --> 00:38:37,486
as many hard disks.

913
00:38:37,486 --> 00:38:40,806
RAID5 and RAID6 are kind of
nice middle grounds with RAID1

914
00:38:41,166 --> 00:38:43,616
or nice variance of RAID1
whereby RAID1 is kind

915
00:38:43,726 --> 00:38:44,746
of pricy, right?

916
00:38:45,056 --> 00:38:48,136
Rather than buy two -- one
hard drive I literally have

917
00:38:48,136 --> 00:38:50,526
to spend twice as much
and get two hard drives.

918
00:38:50,526 --> 00:38:54,946
RAID5 is a little more versatile
whereby I typically have say

919
00:38:54,946 --> 00:38:58,276
three drives, four drives,
five drives, but only one

920
00:38:58,276 --> 00:39:00,026
of them is used for redundancy.

921
00:39:00,396 --> 00:39:03,706
So if I get five,
one terabyte drives,

922
00:39:04,116 --> 00:39:06,536
I have four terabytes
of usable space.

923
00:39:07,096 --> 00:39:09,676
So I'm only sacrificing
one-fifth in that case

924
00:39:09,676 --> 00:39:12,206
of my available disk
capacity whereas

925
00:39:12,206 --> 00:39:16,416
in RAID1 you're sacrificing
one half so 50% of it.

926
00:39:16,606 --> 00:39:18,276
So RAID5, you just
get better economy

927
00:39:18,276 --> 00:39:20,266
at scale whereby
you can grow bigger

928
00:39:20,616 --> 00:39:22,756
and you still have
some redundancy.

929
00:39:22,966 --> 00:39:26,306
So in RAID5 if you have three
or four or five hard drives

930
00:39:26,396 --> 00:39:29,516
in the array, one of them
can die, any of them.

931
00:39:29,516 --> 00:39:31,036
You run to the store
put in a new one

932
00:39:31,086 --> 00:39:32,266
and you haven't lost any data.

933
00:39:32,546 --> 00:39:36,626
RAID6 is even better, what
does RAID6 do, do you think?

934
00:39:37,206 --> 00:39:37,506
Axle?

935
00:39:37,706 --> 00:39:39,156
>> Well I think two
drives can die.

936
00:39:39,156 --> 00:39:42,736
>> Exactly in RAID6 any two
drives can die you still won't

937
00:39:42,736 --> 00:39:44,446
have lost any data, and
so long as you've run

938
00:39:44,446 --> 00:39:45,976
to the store fast
enough and put in one

939
00:39:45,976 --> 00:39:48,516
or both drives again,
you'll be good to go.

940
00:39:48,516 --> 00:39:49,526
Of course the price you pay

941
00:39:49,526 --> 00:39:51,416
with RAID6 is literally
another hard drive

942
00:39:51,766 --> 00:39:53,516
but at least now you can
maybe sleep a bit better

943
00:39:53,516 --> 00:39:56,016
at night knowing that man,
two of my hard drives has

944
00:39:56,016 --> 00:39:58,886
to die before I have to
really worry about this.

945
00:39:59,666 --> 00:40:01,996
So these are really
nice technologies and so

946
00:40:01,996 --> 00:40:04,926
as Axle proposes here, the
upside of using something

947
00:40:04,926 --> 00:40:08,696
like that in whatever file
server were storing our shared

948
00:40:08,696 --> 00:40:11,126
sessions is we can at least
decrease the probability

949
00:40:11,126 --> 00:40:14,566
of downtime at least
related to hard disks.

950
00:40:14,566 --> 00:40:17,806
Unfortunately it still has a
power cord that someone can trip

951
00:40:17,806 --> 00:40:19,386
over or the power
supply could die.

952
00:40:19,676 --> 00:40:21,566
It still has RAM that
could go on the fritz,

953
00:40:21,566 --> 00:40:22,836
a motherboard that could die.

954
00:40:22,836 --> 00:40:24,626
Any number of things
could still happen,

955
00:40:24,936 --> 00:40:27,176
but at least we can
throw redundancy inside

956
00:40:27,176 --> 00:40:28,876
of the confines of
a single server

957
00:40:28,876 --> 00:40:31,316
and this can definitely
help with our uptime

958
00:40:31,476 --> 00:40:32,596
and with our robustness.

959
00:40:32,596 --> 00:40:35,146
And indeed with actual
servers that you would buy

960
00:40:35,226 --> 00:40:37,526
for a data center,
not so much of a home.

961
00:40:37,836 --> 00:40:39,356
It's very common for computers

962
00:40:39,356 --> 00:40:42,616
to have not only multiple
hard drives and lots

963
00:40:42,616 --> 00:40:44,226
of multiple banks of RAM,

964
00:40:44,606 --> 00:40:48,576
and they would often have
multiple power supplies as well.

965
00:40:48,576 --> 00:40:50,466
And it's actually a really
cool technology there too.

966
00:40:50,466 --> 00:40:51,966
If one of your power
supplies dies,

967
00:40:52,196 --> 00:40:53,476
you can literally pull it out.

968
00:40:53,626 --> 00:40:55,686
The machine keeps running,
you put in a new one

969
00:40:55,946 --> 00:40:57,456
and then it spreads the amperage

970
00:40:57,456 --> 00:41:00,806
across two power supplies once
both are back up and running,

971
00:41:00,976 --> 00:41:02,246
all very hot swappable.

972
00:41:02,586 --> 00:41:04,666
Amazing technology these
days and as an aside,

973
00:41:04,666 --> 00:41:06,076
if you still own a
desktop computer,

974
00:41:06,076 --> 00:41:08,166
there is no reason you
shouldn't use RAID these days.

975
00:41:08,166 --> 00:41:11,586
It is just very good practice
since it will allow you

976
00:41:11,616 --> 00:41:15,136
to avoid downtime and data
loss with higher probability.

977
00:41:16,246 --> 00:41:20,366
Okay, but someone tripped over
the power code, someone tripped

978
00:41:20,366 --> 00:41:22,076
over both power codes
in the case

979
00:41:22,076 --> 00:41:23,306
of redundant power supplies.

980
00:41:23,306 --> 00:41:25,396
So Axle's solution and even mine

981
00:41:25,396 --> 00:41:28,096
with redundant power supplies
hasn't solved the problem

982
00:41:28,096 --> 00:41:30,136
of shared storage becoming all

983
00:41:30,136 --> 00:41:32,786
of a sudden a single
point of failure.

984
00:41:33,296 --> 00:41:34,796
So what else could we do

985
00:41:35,136 --> 00:41:38,506
to still get the
property of shared state.

986
00:41:38,506 --> 00:41:40,806
So it doesn't matter what
backend server I end up on

987
00:41:41,276 --> 00:41:45,016
but I instead get -- I
still get to the ability

988
00:41:45,086 --> 00:41:47,126
to suffer some downtime.

989
00:41:47,346 --> 00:41:49,866
Well shared storage can come
in a bunch of different ways,

990
00:41:49,866 --> 00:41:52,346
so we talked really about
things as a file server

991
00:41:52,586 --> 00:41:54,916
but this can be incarnated
with very specific technology.

992
00:41:54,916 --> 00:41:57,126
And just to rattle them off even
though we won't talk about them

993
00:41:57,126 --> 00:42:01,346
in much technical detail, Fiber
Channel, FC, is a very fast,

994
00:42:01,456 --> 00:42:04,506
a very expensive technology
that you can use in offices,

995
00:42:04,506 --> 00:42:06,396
in data centers not
so much the home

996
00:42:06,756 --> 00:42:09,636
to provide very fast shared
storage across server.

997
00:42:09,636 --> 00:42:11,906
So that's just one type of
file server if you will.

998
00:42:11,906 --> 00:42:17,196
ISCSI is another technology
that uses IP, Internet Protocol,

999
00:42:17,336 --> 00:42:20,346
and uses Ethernet cables to
exchange data with servers

1000
00:42:20,346 --> 00:42:22,966
so that's a nice somewhat
cheaper way of exchange --

1001
00:42:22,966 --> 00:42:29,416
of having a shared file server
that can be used by multiple --

1002
00:42:29,416 --> 00:42:31,456
actually in the case of
ISCSI you typically use it

1003
00:42:31,456 --> 00:42:32,436
with it single servers.

1004
00:42:32,716 --> 00:42:34,816
So let me re-track that,
that is not a solution

1005
00:42:34,816 --> 00:42:36,196
to our current cookie problem.

1006
00:42:36,516 --> 00:42:37,876
But what about MySQL?

1007
00:42:37,876 --> 00:42:40,596
All right, we use that
for a couple of weeks,

1008
00:42:41,206 --> 00:42:43,266
MySQL seems to be
a nice candidate

1009
00:42:43,266 --> 00:42:45,416
because it's already a
separate server potentially.

1010
00:42:45,676 --> 00:42:49,786
Could not the backend servers
just write their session objects

1011
00:42:49,786 --> 00:42:52,146
to a database, they
definitely could.

1012
00:42:52,636 --> 00:42:55,326
So just because you're
storing things in --

1013
00:42:55,526 --> 00:42:58,186
just because we usually
store things like user data

1014
00:42:58,186 --> 00:43:00,006
and user generated
and a database,

1015
00:43:00,006 --> 00:43:01,476
doesn't mean we can't
store metadata

1016
00:43:01,476 --> 00:43:03,696
like our cookie information
as well,

1017
00:43:04,096 --> 00:43:05,646
or that too comes
from users though.

1018
00:43:05,846 --> 00:43:08,516
NFS, Network File System,
this is just a protocol

1019
00:43:08,516 --> 00:43:11,566
that you can use to implement
the idea that Axle proposed

1020
00:43:11,566 --> 00:43:13,086
of a shared file system.

1021
00:43:13,086 --> 00:43:14,516
It just means you've
got one server

1022
00:43:14,676 --> 00:43:16,316
and you're exposing
your hard disk

1023
00:43:16,606 --> 00:43:17,986
to multiple other computers.

1024
00:43:18,526 --> 00:43:23,966
But again, we haven't really
solved the problem of downtime,

1025
00:43:24,276 --> 00:43:27,866
so what's the most obvious
way of mitigating the risk

1026
00:43:28,016 --> 00:43:30,186
that your single file
server will go down?

1027
00:43:30,186 --> 00:43:30,346
Axle?

1028
00:43:30,906 --> 00:43:36,166
>> Keep your copy of the
sessions in another location.

1029
00:43:36,166 --> 00:43:36,456
>> Good, right?

1030
00:43:36,456 --> 00:43:38,306
If it's not -- if
you don't have --

1031
00:43:38,676 --> 00:43:40,876
if you're worried about
the one file server going

1032
00:43:40,876 --> 00:43:43,466
down well the obvious
solution even though money

1033
00:43:43,466 --> 00:43:46,236
and some technical
complexity will just get two.

1034
00:43:46,356 --> 00:43:49,296
Now somehow you have to
figure out how to sink the two

1035
00:43:49,616 --> 00:43:52,966
so that one has a copy of the
other's data and vice versa.

1036
00:43:52,966 --> 00:43:55,766
So let's actually come back to
that issue, it's generally known

1037
00:43:55,766 --> 00:43:58,456
as replication but it
is something we can

1038
00:43:58,576 --> 00:43:59,886
potentially achieve.

1039
00:44:00,166 --> 00:44:07,076
But before we segue to
distribution of things,

1040
00:44:07,126 --> 00:44:10,336
let's finish out this
Load Balancer question.

1041
00:44:10,336 --> 00:44:12,696
So how do you go about
implementing this black box?

1042
00:44:12,696 --> 00:44:14,556
Well these days you actually
have a bunch of options.

1043
00:44:14,996 --> 00:44:17,466
In software, you can do
things relatively easily

1044
00:44:17,466 --> 00:44:20,276
with a browser, pointing
and clicking using things

1045
00:44:20,276 --> 00:44:23,086
like amazons, elastic load
balancer, or a scenario

1046
00:44:23,086 --> 00:44:24,606
for which we'll talk
about a bit later.

1047
00:44:24,606 --> 00:44:27,256
HAProxy, High Availability
Proxy,

1048
00:44:27,256 --> 00:44:29,856
is free open source software
that you can run on a server

1049
00:44:30,096 --> 00:44:32,096
that can do a load
balancing as well using any --

1050
00:44:32,096 --> 00:44:35,206
either of the characteristics we
discussed earlier around Robin

1051
00:44:35,206 --> 00:44:37,116
or actually taking load
into account somehow.

1052
00:44:37,466 --> 00:44:41,026
Linux Virtual Server, LVS
is another free software,

1053
00:44:41,026 --> 00:44:43,036
a piece of software you
can use and the world

1054
00:44:43,036 --> 00:44:46,516
of hardware people have made big
business out of Load Balancers,

1055
00:44:46,516 --> 00:44:48,246
Barracuda, Cisco, Citrix,

1056
00:44:48,286 --> 00:44:50,216
F5 are some of the most
popular vendors here.

1057
00:44:50,766 --> 00:44:54,316
Most of whom are atrociously
overpriced for what they do.

1058
00:44:54,316 --> 00:44:57,786
So case and point like
Citrix is a popular company

1059
00:44:57,786 --> 00:44:59,466
that sells Load Balancers
take a guess

1060
00:44:59,466 --> 00:45:02,656
as to what a Load Balancer
might cost you these days?

1061
00:45:02,656 --> 00:45:03,986
It's a highly variable range

1062
00:45:04,026 --> 00:45:06,136
but there's different
models, but take a guess?

1063
00:45:06,856 --> 00:45:08,606
How much of that
black buck cost?

1064
00:45:08,606 --> 00:45:08,896
Isaac?

1065
00:45:08,896 --> 00:45:09,306
>> Thousands.

1066
00:45:10,326 --> 00:45:12,906
>> Definitely in the
thousands, indeed.

1067
00:45:12,906 --> 00:45:18,286
In fact, we have a small
one, so to speak --

1068
00:45:18,286 --> 00:45:20,406
small one relatively
speaking on campus

1069
00:45:20,406 --> 00:45:24,776
that was $20,000 and guess what?

1070
00:45:25,276 --> 00:45:26,116
That one is cheap.

1071
00:45:27,076 --> 00:45:28,576
So you can literally
spend on these kind

1072
00:45:28,576 --> 00:45:31,766
of things granted not, this is
what the per cost the weight

1073
00:45:31,766 --> 00:45:33,316
you've read after the
semester ends today.

1074
00:45:33,746 --> 00:45:37,256
But a $100,000 for
a Load Balancer

1075
00:45:37,256 --> 00:45:39,996
or even generally a pair of
Load Balancers so that either

1076
00:45:39,996 --> 00:45:42,556
of them can die and the
other one can stay alive.

1077
00:45:42,556 --> 00:45:44,166
So in the world of
enterprise hardware,

1078
00:45:44,336 --> 00:45:45,386
these ideas we're talking

1079
00:45:45,386 --> 00:45:47,566
about are ridiculously
priced typically

1080
00:45:47,566 --> 00:45:49,856
because of support
contracts and the like.

1081
00:45:50,116 --> 00:45:52,786
So just realize software
is number one on the list

1082
00:45:52,826 --> 00:45:54,186
because there are other ways

1083
00:45:54,186 --> 00:45:56,146
to achieve this much
more inexpensively.

1084
00:45:56,146 --> 00:46:00,586
Indeed for years one of the
courses I teach we used HAProxy

1085
00:46:00,796 --> 00:46:02,906
to balance load because
it was so relatively easy

1086
00:46:02,906 --> 00:46:04,756
to set up and 100$ free.

1087
00:46:04,886 --> 00:46:07,706
So realize the same ideas
can be both bought and set

1088
00:46:07,706 --> 00:46:11,696
up on your own quite
readily these days.

1089
00:46:12,686 --> 00:46:15,856
All right let's pause here

1090
00:46:15,856 --> 00:46:17,596
and when we come back we'll
take a look at some issues

1091
00:46:17,596 --> 00:46:19,956
of like caching, of
replication in databases

1092
00:46:19,956 --> 00:46:22,346
and also how we can
speed up PHP a bit.

1093
00:46:22,486 --> 00:46:27,186
Let's take our five
minute break here.

1094
00:46:27,386 --> 00:46:29,486
All right we're back
and I almost forgot,

1095
00:46:29,486 --> 00:46:32,436
we have one other
solution to this problem

1096
00:46:32,516 --> 00:46:34,156
of the need for sticky sessions.

1097
00:46:34,206 --> 00:46:35,506
Sticky sessions meaning

1098
00:46:35,736 --> 00:46:39,636
that when you visit a
website multiple times,

1099
00:46:39,636 --> 00:46:42,456
your session is somehow
preserved even

1100
00:46:42,456 --> 00:46:44,156
if there're multiple
backend servers.

1101
00:46:44,496 --> 00:46:48,656
So shared storage was the idea
we really vetted quite a bit

1102
00:46:48,656 --> 00:46:51,576
and we didn't quite get
to a perfect solution

1103
00:46:51,906 --> 00:46:55,416
since even though we
factored out the storage,

1104
00:46:55,706 --> 00:46:57,966
and put everyone's
cookies or session objects

1105
00:46:57,966 --> 00:47:00,636
on the same server, it feels
like we need some redundancy

1106
00:47:00,636 --> 00:47:02,126
but we'll come back
to that in the context

1107
00:47:02,126 --> 00:47:03,436
of MySQL in just a bit.

1108
00:47:03,436 --> 00:47:04,786
But what about cookies?

1109
00:47:05,416 --> 00:47:09,566
I propose that cookies
themselves could offer a

1110
00:47:09,566 --> 00:47:12,916
solution to the problem
of sticky sessions

1111
00:47:13,256 --> 00:47:15,136
and again sticky
sessions means even

1112
00:47:15,136 --> 00:47:19,866
if you visit a website multiple
times, you're still going to end

1113
00:47:19,866 --> 00:47:21,916
up with the same session
object, or more specifically,

1114
00:47:21,916 --> 00:47:25,056
you're still going to end up on
the same backend server, Axle?

1115
00:47:25,056 --> 00:47:28,376
>> Either of the -- store

1116
00:47:28,376 --> 00:47:34,596
which server you want
to go to in a cookie.

1117
00:47:34,776 --> 00:47:34,866
>> Okay.

1118
00:47:34,866 --> 00:47:36,756
>> Or store everything
in cookies

1119
00:47:36,916 --> 00:47:40,146
but that's not a good solution.

1120
00:47:41,836 --> 00:47:43,726
>> Store or -- okay, yes,
so storing everything

1121
00:47:43,726 --> 00:47:44,906
in cookies is probably bad

1122
00:47:45,216 --> 00:47:48,146
because one then it's really
starting to violate privacy

1123
00:47:48,146 --> 00:47:50,316
because rather than store a big
a key, you're going to store

1124
00:47:50,316 --> 00:47:53,016
like the ISPNs of all the
books in your shopping cart

1125
00:47:53,016 --> 00:47:55,066
and that might be fine but
feels like your roommates

1126
00:47:55,066 --> 00:47:57,916
and family members don't need
to know what is in your cookies.

1127
00:47:57,916 --> 00:48:00,366
Moreover cookies
typically have a finite size

1128
00:48:00,706 --> 00:48:02,366
of a few kilobytes, so
there's definitely going

1129
00:48:02,366 --> 00:48:04,736
to be circumstances in which you
just can't fit everything you

1130
00:48:04,736 --> 00:48:05,876
want to in the cookie.

1131
00:48:05,876 --> 00:48:09,256
So you know an interesting
idea but probably not the best,

1132
00:48:09,256 --> 00:48:12,336
so you could store the ID
of a server in a cookie

1133
00:48:12,566 --> 00:48:15,176
so that the user, the second,
and third, and fourth times,

1134
00:48:15,176 --> 00:48:17,626
they visit your website
as by following links

1135
00:48:17,936 --> 00:48:21,546
or coming back some other
time, they are going

1136
00:48:21,546 --> 00:48:23,776
to present the equivalent
of a hand stamp saying,

1137
00:48:23,776 --> 00:48:27,246
"Hey I was on backend server
1, send me there again."

1138
00:48:27,806 --> 00:48:30,286
So that's a pretty nice
idea, there is one,

1139
00:48:30,976 --> 00:48:33,676
at least one downside
here, what do you like --

1140
00:48:33,676 --> 00:48:35,576
what do you not like
potentially about this idea

1141
00:48:35,576 --> 00:48:38,716
of storing inner cookie that
gets put on user's browser,

1142
00:48:38,716 --> 00:48:41,906
that they subsequently transmit
back to you the number of,

1143
00:48:41,906 --> 00:48:48,106
or the ID of the server to
which they should be sent.

1144
00:48:48,296 --> 00:48:48,556
Axle?

1145
00:48:48,686 --> 00:48:49,906
>> Well from runner expiration.

1146
00:48:50,506 --> 00:48:51,566
>> Expiration in what sense?

1147
00:48:52,196 --> 00:48:53,596
>> All of cookie
expires after [inaudible]

1148
00:48:54,356 --> 00:48:57,336
>> Okay, so eventually cookie
is going to expire though,

1149
00:48:57,336 --> 00:49:00,916
we saw a couple lectures just
ago, we couldn't make it expire

1150
00:49:00,916 --> 00:49:03,966
in 10 years if we really
wanted to and frankly,

1151
00:49:03,966 --> 00:49:06,276
we're never going to avoid that
even if we had a single server,

1152
00:49:06,276 --> 00:49:07,976
cookies could eventually expire.

1153
00:49:07,976 --> 00:49:10,256
So at least that's not a new
problem, so I'm not too worried

1154
00:49:10,256 --> 00:49:14,176
about expiration now, because
that's not a problem new

1155
00:49:14,386 --> 00:49:16,946
to us simply because
of load balancing.

1156
00:49:17,466 --> 00:49:22,726
Does anything not feel
right about storing the idea

1157
00:49:22,726 --> 00:49:23,936
of the server in the cookie?

1158
00:49:23,936 --> 00:49:31,076
>> What if the IP changes?

1159
00:49:32,476 --> 00:49:35,216
>> Yeah, so if we just
put like the backend IP

1160
00:49:35,296 --> 00:49:37,866
so the private IP address
in the cookie, you know what

1161
00:49:37,866 --> 00:49:40,786
of the IP changes, what if
that -- what if the IP changes,

1162
00:49:40,786 --> 00:49:42,956
so that's a little
problematic and it's also one

1163
00:49:42,956 --> 00:49:46,016
of these principle things like,
you don't really need to reveal

1164
00:49:46,066 --> 00:49:48,306
to the world what your
IP address scheme is,

1165
00:49:48,396 --> 00:49:50,746
it's not necessarily
something they could exploit

1166
00:49:50,746 --> 00:49:53,406
but it's just the whole world
doesn't need to know that.

1167
00:49:53,676 --> 00:49:55,826
Moreover we can implement
the same idea

1168
00:49:56,036 --> 00:49:57,926
by still storing a cookie
on the user's computer

1169
00:49:57,926 --> 00:49:59,566
but why don't we
take the PHP approach

1170
00:49:59,566 --> 00:50:01,626
of let's just store
a big random number

1171
00:50:01,806 --> 00:50:03,916
and then have the
load balancer remember

1172
00:50:04,016 --> 00:50:05,976
that that big random number
belongs to server one.

1173
00:50:06,256 --> 00:50:08,196
And this other big
random number belongs

1174
00:50:08,196 --> 00:50:10,336
to server two and so forth.

1175
00:50:10,366 --> 00:50:12,376
So a little more work
on the load balancer

1176
00:50:12,376 --> 00:50:16,046
but in this way then, we're
really not putting any states

1177
00:50:16,526 --> 00:50:19,626
that might change or might
be a little privacy revealing

1178
00:50:19,626 --> 00:50:21,396
on the actual user's computer.

1179
00:50:21,566 --> 00:50:24,876
Moreover we also take away
the ability for them to spoof

1180
00:50:24,926 --> 00:50:28,236
that cookie just to get
access to some other server.

1181
00:50:28,236 --> 00:50:29,516
Now whether or not
they could anything

1182
00:50:29,516 --> 00:50:33,596
with that trick is unclear
but at least we take away

1183
00:50:33,596 --> 00:50:35,656
that ability altogether
so there's no surprises.

1184
00:50:35,656 --> 00:50:39,056
All right, so cookies indeed are
something that these black boxes

1185
00:50:39,056 --> 00:50:40,236
of -- so load balancers tend

1186
00:50:40,236 --> 00:50:42,226
to do whereby you
can configure them

1187
00:50:42,596 --> 00:50:45,176
to insert a cookie
themselves, it doesn't just have

1188
00:50:45,176 --> 00:50:47,366
to be a backend web server
that generates cookies,

1189
00:50:47,566 --> 00:50:50,576
the load balancer similarly
could be inserting a cookie

1190
00:50:50,976 --> 00:50:52,346
with the set cookie header

1191
00:50:52,586 --> 00:50:56,116
that the end user then
subsequently sends back

1192
00:50:56,346 --> 00:50:58,486
so that we can remember
what backend server

1193
00:50:58,486 --> 00:50:59,546
to send the user to.

1194
00:50:59,716 --> 00:51:01,996
Now if the user has cookies
disabled well then this whole

1195
00:51:01,996 --> 00:51:04,586
system breaks down but
again so does a lot

1196
00:51:04,586 --> 00:51:07,316
of functionality we've
discussed this far this semester

1197
00:51:07,716 --> 00:51:09,306
but there are sometimes
some workarounds.

1198
00:51:09,976 --> 00:51:13,326
So a word on PHP, PHP in
an interpreted languages is

1199
00:51:13,326 --> 00:51:15,686
in general tend to get a
bad rep for performance

1200
00:51:15,906 --> 00:51:17,846
because they tend not to
be as a high as performing

1201
00:51:17,846 --> 00:51:20,496
as a compiled language like
C plus plus or C or the like.

1202
00:51:21,206 --> 00:51:24,196
However there are some ways to
mitigate this, there is notion

1203
00:51:24,196 --> 00:51:29,316
of PHP Acceleration whereby
you can run a PHP program,

1204
00:51:29,646 --> 00:51:33,786
the source code through php.exe,
the interpreter on the system

1205
00:51:34,016 --> 00:51:37,666
and it turns out that PHP does
typically compile that file

1206
00:51:37,666 --> 00:51:39,836
in a sense, down to something
that's more efficiently

1207
00:51:39,836 --> 00:51:41,866
executed, much like
Java complied something

1208
00:51:41,866 --> 00:51:43,626
down to something
called Bi code.

1209
00:51:44,206 --> 00:51:47,806
But you typically --
PHP throws the results

1210
00:51:48,206 --> 00:51:49,526
of that compilation away.

1211
00:51:50,066 --> 00:51:52,136
Whereby it does it
again and again

1212
00:51:52,136 --> 00:51:55,566
for every subsequent request,
however with very simple --

1213
00:51:55,566 --> 00:51:58,106
with relatively straight --

1214
00:51:58,106 --> 00:52:00,266
forward and free the
available software,

1215
00:52:00,456 --> 00:52:02,846
you can install a
PHP Accelerator here

1216
00:52:02,846 --> 00:52:06,186
in just four possibilities
that essentially eliminate

1217
00:52:06,186 --> 00:52:09,386
that discarding of
the PHP Opcodes

1218
00:52:09,566 --> 00:52:10,906
and instead keep them around.

1219
00:52:10,906 --> 00:52:13,386
So in other words the first
time someone visits your site,

1220
00:52:13,646 --> 00:52:16,836
the PHP file is going to be
interpreted and some Opcodes

1221
00:52:16,836 --> 00:52:19,546
for performance generated
but they are not going

1222
00:52:19,546 --> 00:52:21,556
to be thrown away
because the next time you

1223
00:52:21,556 --> 00:52:24,676
or someone else visits the site,
that PHP file is not going have

1224
00:52:24,676 --> 00:52:26,566
to be reparsed and
reinterpreted,

1225
00:52:26,776 --> 00:52:29,566
the Opcodes are just going to be
executed, so you get the benefit

1226
00:52:29,566 --> 00:52:30,706
of some added performance.

1227
00:52:30,886 --> 00:52:33,146
Now the only got you is
if you ever change any

1228
00:52:33,146 --> 00:52:39,316
of your dot php files, you have
to throw away the cached Opcodes

1229
00:52:39,706 --> 00:52:42,766
but these various tools
typically do that for you.

1230
00:52:42,806 --> 00:52:45,986
Python has a similar mechanism
where you'll get dot py files

1231
00:52:45,986 --> 00:52:47,026
or your source code files

1232
00:52:47,346 --> 00:52:50,886
but dot pyc files are
the complied versions

1233
00:52:50,886 --> 00:52:52,536
that can be executed
more quickly,

1234
00:52:52,776 --> 00:52:54,586
so the same idea
is at play here.

1235
00:52:54,586 --> 00:52:57,946
So this one of these things
that is relatively easy and free

1236
00:52:57,946 --> 00:53:01,396
to enable, and gives you
all the more performance,

1237
00:53:01,616 --> 00:53:02,546
specifically the ability

1238
00:53:02,546 --> 00:53:05,506
to handle all the more requests
per seconds in the context

1239
00:53:05,506 --> 00:53:07,136
of a PHP based website.

1240
00:53:07,686 --> 00:53:08,896
So what about caching too?

1241
00:53:08,896 --> 00:53:11,056
Caching in general is a
great thing, it solves some

1242
00:53:11,056 --> 00:53:13,836
of our DNS concerns early
on but it introduced others

1243
00:53:14,176 --> 00:53:18,266
because caching can be a bad
thing if some value has changed

1244
00:53:18,266 --> 00:53:19,266
but you have the old one.

1245
00:53:19,656 --> 00:53:21,726
But Caching can be
implemented in the context

1246
00:53:21,726 --> 00:53:25,266
of dynamic websites in a few
different ways, so I propose

1247
00:53:25,266 --> 00:53:27,266
that through HTML,
through MySQL,

1248
00:53:27,396 --> 00:53:28,606
and through something
called memcached,

1249
00:53:28,606 --> 00:53:30,926
we can achieve some
caching benefits here.

1250
00:53:31,286 --> 00:53:35,926
So this is a screen shot of
one of the most 1990s websites

1251
00:53:35,926 --> 00:53:38,216
out there and this was not
even taken in the 1990s,

1252
00:53:38,216 --> 00:53:40,626
this was taken a couple of
years ago and I just visited

1253
00:53:40,626 --> 00:53:43,466
out of curiosity craigslist
today, it still looks the same.

1254
00:53:43,466 --> 00:53:46,046
So what's interesting
about Craigslist is

1255
00:53:46,046 --> 00:53:49,666
that it is a dynamic website
and that you can fill out a form

1256
00:53:49,666 --> 00:53:53,006
and post a for sale ad or
roommate ad or of the like,

1257
00:53:53,006 --> 00:53:56,006
and the website does
actually change but it's --

1258
00:53:56,006 --> 00:53:58,326
if we zoom in on this and it's
going to be a little blurry

1259
00:53:58,326 --> 00:54:00,866
because of the screen
shot, the URL that's

1260
00:54:00,866 --> 00:54:05,476
up there is actually dot html
even though it's barely readable

1261
00:54:05,476 --> 00:54:06,326
at this resolution.

1262
00:54:06,786 --> 00:54:07,906
Which is to suggest

1263
00:54:08,026 --> 00:54:11,796
that craigslist is
apparently accepting user input

1264
00:54:11,796 --> 00:54:14,856
through forms for
instance whoever wrote

1265
00:54:15,186 --> 00:54:17,536
up this job advertisement
sometime ago

1266
00:54:17,886 --> 00:54:19,556
but then Craigslist
it's speeding it

1267
00:54:19,556 --> 00:54:22,786
out as a dot html
file as opposed

1268
00:54:22,786 --> 00:54:26,046
to storing it where or in what?

1269
00:54:26,046 --> 00:54:26,206
Axle?

1270
00:54:26,246 --> 00:54:32,776
>> I suppose to storing the
data that [inaudible] page

1271
00:54:32,776 --> 00:54:34,856
in the database then
printing it in PHP.

1272
00:54:35,016 --> 00:54:37,446
>> Yeah so this is in stark
contrast of what we've done

1273
00:54:37,446 --> 00:54:41,286
for project zero, project
one, we're using PHP

1274
00:54:41,346 --> 00:54:44,306
as the backend whereby you store
data like this, like server side

1275
00:54:44,306 --> 00:54:46,726
and maybe an XML file
or more realistically

1276
00:54:46,976 --> 00:54:50,636
in a MySQL database or similar,
and then you generate a page

1277
00:54:50,636 --> 00:54:51,816
like this dynamically.

1278
00:54:52,256 --> 00:54:55,086
So why is Craigslist
doing this apparently?

1279
00:54:56,386 --> 00:54:57,806
Could just be they
are stuck in the '90s

1280
00:54:57,896 --> 00:55:00,776
but there's compelling
reason too, Axle?

1281
00:55:01,116 --> 00:55:05,086
>> Well if they store the actual
HTML file then they don't have

1282
00:55:05,176 --> 00:55:07,146
to regenerate it every
time they it's visited.

1283
00:55:07,386 --> 00:55:09,586
>> Yeah exactly, if they are
storing the html file they just

1284
00:55:09,586 --> 00:55:12,186
don't have to regenerate it
every time it's re visited,

1285
00:55:12,656 --> 00:55:15,126
so this itself is
caching, it's not caching

1286
00:55:15,126 --> 00:55:18,316
in any particularly fancy way,
you're just generating the html

1287
00:55:18,316 --> 00:55:21,056
and saving it in something
called like something.html

1288
00:55:21,056 --> 00:55:22,066
and storing it on disk.

1289
00:55:22,066 --> 00:55:24,516
And the upside of this is
that web servers like Apache,

1290
00:55:24,516 --> 00:55:27,536
you're really, really, really
good and fast at just spitting

1291
00:55:27,536 --> 00:55:31,416
out bits, just spitting raw
static content like a GIF,

1292
00:55:31,416 --> 00:55:35,086
a JPEG, and HTML file but the
performance optimization these

1293
00:55:35,086 --> 00:55:38,566
days generally relate to the
languages like PHP, and PYTHON,

1294
00:55:38,566 --> 00:55:40,536
and RUBY where you're trying
to fine-tune performance,

1295
00:55:40,876 --> 00:55:44,806
but if all you have to do is
respond to a TCPIP request

1296
00:55:45,086 --> 00:55:46,396
with a bunch of bytes from disk.

1297
00:55:46,736 --> 00:55:47,756
That's relatively straight --

1298
00:55:47,786 --> 00:55:50,006
forward these days and so
they're taking advantage

1299
00:55:50,006 --> 00:55:53,246
of the performance presumably
of serving up static content

1300
00:55:53,666 --> 00:55:55,046
but this comes at a cost.

1301
00:55:55,636 --> 00:55:59,476
What's the downside of this
file based caching approach?

1302
00:56:03,016 --> 00:56:04,626
Someone else?

1303
00:56:06,236 --> 00:56:10,306
Nothing we've this far is
sort of a complete win,

1304
00:56:10,456 --> 00:56:11,776
there's always a got you, Louis?

1305
00:56:12,126 --> 00:56:12,316
>> Space.

1306
00:56:12,906 --> 00:56:16,356
>> Okay so space, so we're
storing it on disk and internet,

1307
00:56:16,356 --> 00:56:17,906
if you either post
it on Craigslist,

1308
00:56:18,136 --> 00:56:20,146
they are also storing it
somewhere in a database

1309
00:56:20,146 --> 00:56:22,426
because they do let
go back and edit it,

1310
00:56:22,996 --> 00:56:25,156
it's just Craigslist
is one of the sites

1311
00:56:25,156 --> 00:56:28,266
where reads are probably
much more common than writes.

1312
00:56:28,436 --> 00:56:29,846
Indeed when people
visit craigslist,

1313
00:56:29,846 --> 00:56:32,506
they are probably flipping
through reading pages as opposed

1314
00:56:32,506 --> 00:56:34,956
to posting lots, and
lots, and lots of ads all

1315
00:56:34,956 --> 00:56:38,336
at the same time, so there
some redundancy there

1316
00:56:38,466 --> 00:56:39,236
that's unfortunate.

1317
00:56:39,446 --> 00:56:39,796
Axle?

1318
00:56:39,796 --> 00:56:47,776
>> Yeah that to -- just
to build upon that,

1319
00:56:48,026 --> 00:56:56,446
you will be storing lots of
code that you use on every page

1320
00:56:56,566 --> 00:56:59,976
over again, plus it's not
a very elegant solution

1321
00:56:59,976 --> 00:57:04,166
to have it fixed on your
server containing 10,000 files.

1322
00:57:04,166 --> 00:57:06,836
>> Okay good so there's
redundancy while actually all

1323
00:57:06,896 --> 00:57:09,676
these thousands of files,

1324
00:57:09,676 --> 00:57:12,066
there's redundancy too
just in the basic stuff.

1325
00:57:12,066 --> 00:57:14,376
Like you have the same HTML
tag, the same body tag,

1326
00:57:14,376 --> 00:57:17,816
the same link tag, the same
grip tag in every single page

1327
00:57:17,816 --> 00:57:20,306
if they're indeed just
static HTML files.

1328
00:57:20,306 --> 00:57:23,246
So whereas, you get some
benefits of using something

1329
00:57:23,246 --> 00:57:27,426
like PHP and recall MVC
discussion where we factored

1330
00:57:27,426 --> 00:57:29,836
out template code, like
the header and the footer,

1331
00:57:29,836 --> 00:57:31,406
so that we stored
it in one place

1332
00:57:31,406 --> 00:57:32,916
and not thousands of places.

1333
00:57:32,916 --> 00:57:36,506
Craigslist is kind of
sacrificing that feature

1334
00:57:36,506 --> 00:57:37,806
and going with this instead,

1335
00:57:37,806 --> 00:57:40,976
so in the end it's probably
a calculated trade off,

1336
00:57:40,976 --> 00:57:44,496
you get much better performance
presumably from just serving

1337
00:57:44,496 --> 00:57:48,626
up the static content, but the
price you pay is more disk space

1338
00:57:49,566 --> 00:57:51,296
but at the same time, you know

1339
00:57:51,296 --> 00:57:57,556
for a few hundred dollars you
can typically get even bigger

1340
00:57:57,556 --> 00:58:03,136
hard drives these days, so
maybe that's actually the lesser

1341
00:58:03,426 --> 00:58:04,416
of the evils.

1342
00:58:04,556 --> 00:58:09,546
But there is one -- there
is another big got you here,

1343
00:58:09,546 --> 00:58:14,086
if you've generated tens of
thousands of craigslist pages

1344
00:58:14,286 --> 00:58:18,556
that look like this,
what's the implication now

1345
00:58:18,586 --> 00:58:22,036
and maybe why are they
stuck in the '90s?

1346
00:58:22,036 --> 00:58:25,206
>> Well I decide to add -- well
change the background color.

1347
00:58:25,206 --> 00:58:25,356
>> Good

1348
00:58:25,356 --> 00:58:28,656
>> Some change the design
part of it which is necessary

1349
00:58:28,846 --> 00:58:31,736
but you can't do that
without editing each one

1350
00:58:32,016 --> 00:58:35,886
of those ten thousand files
and that could be automatically

1351
00:58:35,886 --> 00:58:39,316
but it's much harder than just
editing the generic template

1352
00:58:39,316 --> 00:58:40,186
that generates it.

1353
00:58:40,186 --> 00:58:42,606
>> Exactly, if you want
to change the aesthetics

1354
00:58:42,606 --> 00:58:44,486
of the page and add a
background color, change the CSS

1355
00:58:44,576 --> 00:58:46,326
or make it the font other
than Times New Roman,

1356
00:58:46,326 --> 00:58:49,816
it's nontrivial now because
assuming this is a fully intact

1357
00:58:49,976 --> 00:58:53,426
HTML file with no server
side include mechanism,

1358
00:58:53,716 --> 00:58:58,276
no like require mechanism
like you have in PHP.

1359
00:58:58,276 --> 00:59:01,076
You have to now change the
background color in tens

1360
00:59:01,186 --> 00:59:03,416
of thousands of files,
unless maybe you at least put

1361
00:59:03,416 --> 00:59:04,856
in a CSS file but even then

1362
00:59:04,856 --> 00:59:07,486
if it's a less trivial
change than color.

1363
00:59:08,096 --> 00:59:10,196
Suppose you want to restructure
the HTML of the page,

1364
00:59:10,196 --> 00:59:14,496
then you really have to do
a massive find and replace

1365
00:59:14,496 --> 00:59:21,356
or more realistically probably
regenerate all ten thousand plus

1366
00:59:21,356 --> 00:59:25,886
pages and I'm -- we latch on
to ten thousand arbitrarily

1367
00:59:25,886 --> 00:59:27,576
but there's a lot of
pages in this case.

1368
00:59:27,576 --> 00:59:29,586
So upsides and downsides,
they're one of the few people

1369
00:59:29,586 --> 00:59:31,706
in the internet who do
this particular approach

1370
00:59:31,736 --> 00:59:32,456
but it does have some value.

1371
00:59:32,486 --> 00:59:33,956
And I think the last time I
read up on statics they get

1372
00:59:33,986 --> 00:59:35,246
by with relatively little
hardware as a result

1373
00:59:35,276 --> 00:59:36,026
which is definitely compelling.

1374
00:59:36,056 --> 00:59:37,706
So MySQL query cache, this is
a mechanism that we didn't use,

1375
00:59:37,736 --> 00:59:39,296
but it's so easily enabled on
a typical server with MySQL,

1376
00:59:39,326 --> 00:59:40,736
there's a file called my.cnf
for your configuration file.

1377
00:59:40,766 --> 00:59:41,666
And you can simply
add a directive

1378
00:59:41,696 --> 00:59:43,316
like query cached type equals
1, and then re start the server

1379
00:59:43,346 --> 00:59:44,876
to enable the query cache which
pretty much does what it says.

1380
00:59:44,906 --> 00:59:46,406
The -- if you execute a command
like select foo from bars

1381
00:59:46,436 --> 00:59:47,516
where bars equal 1,
2, 3, that would be --

1382
00:59:47,546 --> 00:59:48,656
could be slow the first
time you execute it

1383
00:59:48,686 --> 00:59:50,336
if you don't have an index, or
if you have a really huge table.

1384
00:59:50,366 --> 00:59:51,956
But the next time you execute
it, if the query cache is on

1385
00:59:51,986 --> 00:59:53,336
and if that row hasn't
changed, the response is going

1386
00:59:53,366 --> 00:59:54,116
to come back much more quickly.

1387
00:59:54,146 --> 00:59:55,166
So MySQL provides
this kind of caching

1388
00:59:55,196 --> 00:59:56,156
for identically executed queries

1389
00:59:56,186 --> 00:59:56,996
which might certainly
happen a lot

1390
00:59:57,026 --> 00:59:58,406
if the user is navigating
your website going forward

1391
00:59:58,436 --> 00:59:58,976
and back quite a bit.

1392
00:59:59,676 --> 01:00:02,536
Memcached is an even
more powerful mechanism;

1393
01:00:03,116 --> 01:00:05,546
Facebook has made
great use of this

1394
01:00:05,546 --> 01:00:07,096
over the years especially
initially.

1395
01:00:07,396 --> 01:00:10,966
Memcached is a memory cache,
so it is a piece of software,

1396
01:00:10,966 --> 01:00:12,526
a server that you
run on a server,

1397
01:00:12,736 --> 01:00:15,146
it can be on the same
server as your web server,

1398
01:00:15,146 --> 01:00:16,816
it can be on a different
box altogether,

1399
01:00:17,106 --> 01:00:18,846
but it essentially
is a mechanism

1400
01:00:18,846 --> 01:00:21,396
that stores whatever
you want in RAM.

1401
01:00:21,976 --> 01:00:26,056
And it does this with -- in
the PHP context with codes

1402
01:00:26,056 --> 01:00:28,176
like this, so this
Memcached can be used

1403
01:00:28,176 --> 01:00:29,396
by all sorts of languages.

1404
01:00:29,856 --> 01:00:34,196
Here is PHP zone interface
to it and you use Memcached

1405
01:00:34,196 --> 01:00:35,526
as follows, you first connect

1406
01:00:35,566 --> 01:00:38,376
to the Memcached server
using Memcached connect

1407
01:00:38,466 --> 01:00:40,796
which is very similar in
spirit to MySQL connect

1408
01:00:40,876 --> 01:00:42,606
which you might recall
from a few lectures back.

1409
01:00:43,146 --> 01:00:46,016
Then we try in this
example to get a user,

1410
01:00:46,106 --> 01:00:48,556
so the context here is,
it's pretty expensive

1411
01:00:48,556 --> 01:00:51,976
to do select star from
users on my database table

1412
01:00:51,976 --> 01:00:53,846
because I've got millions
of users in this table

1413
01:00:53,846 --> 01:00:56,206
and I'd really rather
not execute

1414
01:00:56,206 --> 01:00:58,186
that query more often
than I have to.

1415
01:00:58,186 --> 01:01:01,606
I'd rather execute it once,
save the results in RAM

1416
01:01:01,796 --> 01:01:04,556
and the next time I need
that user, go into the RAM,

1417
01:01:04,556 --> 01:01:06,546
go into the cache to
get that user rather

1418
01:01:06,546 --> 01:01:07,576
than touching the database.

1419
01:01:07,916 --> 01:01:11,486
So there's this sort of
tier performance objectives,

1420
01:01:11,886 --> 01:01:14,986
disk is slow, right spinning
disks are specially slow,

1421
01:01:15,376 --> 01:01:18,746
but faster serve up so
generally you might want

1422
01:01:18,746 --> 01:01:20,946
to store something instead of
on disk, you want to store it

1423
01:01:20,946 --> 01:01:23,016
in a table that has indexes

1424
01:01:23,356 --> 01:01:25,516
so that you can search
it more quickly.

1425
01:01:25,516 --> 01:01:27,276
For instance think
back to project zero,

1426
01:01:27,276 --> 01:01:30,126
the XML file is relatively
slow but at the same time,

1427
01:01:30,126 --> 01:01:32,766
anytime you wanted to search it,
you had to load it from disk,

1428
01:01:32,766 --> 01:01:34,086
build a dome thanks

1429
01:01:34,136 --> 01:01:37,886
to the simple XML API then
search it, kind of annoying.

1430
01:01:37,886 --> 01:01:39,856
It would be nice if we
could skip the disk step

1431
01:01:40,176 --> 01:01:42,776
so that things would just be
faster and thus, was born MySQL

1432
01:01:42,776 --> 01:01:44,916
on project one, MySQL
is a server

1433
01:01:45,116 --> 01:01:47,686
which means it's always running,
it's using some RAM so in

1434
01:01:47,686 --> 01:01:50,106
that case you have the
ability to execute queries

1435
01:01:50,106 --> 01:01:53,656
on a data that's hopefully
in RAM but if it isn't you

1436
01:01:53,656 --> 01:01:57,286
at least have the opportunity
to define indexes, primary keys,

1437
01:01:57,286 --> 01:02:00,486
unique keys, index fields, so
that at least you can search

1438
01:02:00,486 --> 01:02:03,816
that data more readily than
you can with say XPath in XML.

1439
01:02:04,176 --> 01:02:06,736
So the next step is not
even to use a database

1440
01:02:06,736 --> 01:02:09,826
because database queries
can be expensive relative

1441
01:02:09,826 --> 01:02:12,276
to just a cache which is
just a key value store.

1442
01:02:12,476 --> 01:02:15,046
I want to give you x equals
y and the next time I ask

1443
01:02:15,046 --> 01:02:17,186
for x you give me y
and I want it quick,

1444
01:02:17,676 --> 01:02:19,416
much faster than a
database would return it.

1445
01:02:19,716 --> 01:02:22,296
So here we've gone and connected
to the memory cache daemon,

1446
01:02:22,366 --> 01:02:26,616
the sever, and the second line
I am trying to get something

1447
01:02:26,616 --> 01:02:30,536
from the cache, the arguments
the Memcached get take the first

1448
01:02:30,536 --> 01:02:33,836
argument is a reference
to the cache that you want

1449
01:02:34,536 --> 01:02:38,916
to grab something from, and then
$ ID just represents something

1450
01:02:38,916 --> 01:02:42,216
like 1, 2, 3, the ID of the
user that I want to get.

1451
01:02:42,216 --> 01:02:46,286
If the user is null, what's the
implication apparently, Isaac?

1452
01:02:46,936 --> 01:02:49,996
>> Well you do the
query all over again.

1453
01:02:49,996 --> 01:02:52,996
>> Okay good but in what case
would user be null do you think?

1454
01:02:52,996 --> 01:02:55,906
>> When they don't exist.

1455
01:02:55,986 --> 01:02:58,016
>> Good when they're not in
the cache, when user 1, 2,

1456
01:02:58,016 --> 01:03:00,116
3 or whoever I'm looking
for is not in the cache,

1457
01:03:00,566 --> 01:03:02,586
that variable is going to
be null and so we do this

1458
01:03:02,586 --> 01:03:04,006
IF condition as Isaac said,

1459
01:03:04,206 --> 01:03:06,456
and here there are
somewhat familiar code,

1460
01:03:06,456 --> 01:03:08,906
PDO which relates to
MySQL in our case.

1461
01:03:08,906 --> 01:03:13,566
We connect to the database
using that user and path,

1462
01:03:13,566 --> 01:03:16,606
we then execute the
query function in PDO,

1463
01:03:16,766 --> 01:03:19,946
in this case select star from
users where ID equals ID.

1464
01:03:20,146 --> 01:03:23,386
I'm not using my --
I'm not escaping ID

1465
01:03:23,386 --> 01:03:25,826
because in this case, I'm
assuming I know it's an integer

1466
01:03:25,826 --> 01:03:28,186
so it's not a dangerous
string just to be clear.

1467
01:03:28,456 --> 01:03:31,366
Then I'm calling fetch to
get back an associative array

1468
01:03:31,706 --> 01:03:34,726
of my data, the users
name, email address, ID,

1469
01:03:34,726 --> 01:03:36,286
whatever else I've
got in my database

1470
01:03:36,286 --> 01:03:40,846
but then the last thing I do
before apparently nothing else

1471
01:03:40,846 --> 01:03:44,466
because this is out of context,
before actually using that user

1472
01:03:44,466 --> 01:03:45,896
for anything, what
am I doing with him?

1473
01:03:46,746 --> 01:03:47,036
Axle?

1474
01:03:47,036 --> 01:03:49,556
>> You're storing it
in the caching memory.

1475
01:03:49,556 --> 01:03:51,756
>> Exactly I'm storing
a key value pair

1476
01:03:51,756 --> 01:03:56,606
in the cache whereby the key is
the user's ID, so this implies

1477
01:03:56,646 --> 01:03:59,446
that there's an ID field in
the user object that came back

1478
01:03:59,446 --> 01:04:01,536
from the database,
from this line here

1479
01:04:01,766 --> 01:04:04,006
and the value is the
user object itself.

1480
01:04:04,366 --> 01:04:08,326
So again a Memcache in this case
is a key value storage mechanism

1481
01:04:08,636 --> 01:04:11,416
and the next time I want to look
up this user, I want to look him

1482
01:04:11,416 --> 01:04:14,296
up by his ID and case in point,

1483
01:04:14,296 --> 01:04:16,746
that's what I did
in line 2 up top.

1484
01:04:17,156 --> 01:04:21,386
Now caches are finite
because RAM is finite

1485
01:04:21,386 --> 01:04:25,516
and even disk is finite, so
what could happen eventually

1486
01:04:25,516 --> 01:04:27,066
with my cache?

1487
01:04:27,946 --> 01:04:30,906
Just by nature of that --
those constraints, Axle?

1488
01:04:31,116 --> 01:04:33,806
>> It gets so big you can't
keep it on the machine.

1489
01:04:33,806 --> 01:04:35,076
>> Good so eventually
the cache could get

1490
01:04:35,076 --> 01:04:36,716
so big you can't it
keep on the machines,

1491
01:04:36,716 --> 01:04:40,456
so what would be a reasonable
thing to do at that point,

1492
01:04:40,456 --> 01:04:42,996
when you've run out of RAM
or disk space for cache?

1493
01:04:46,836 --> 01:04:49,766
You're the person implementing
Memcached itself now,

1494
01:04:49,766 --> 01:04:51,846
what do you do in that
case, you could just kind

1495
01:04:51,846 --> 01:04:56,606
of quit unexpected error
but would kind of be bad

1496
01:04:57,446 --> 01:04:58,876
and completely unnecessary.

1497
01:04:59,386 --> 01:05:04,016
What could you do?

1498
01:05:04,256 --> 01:05:04,886
Isaac?

1499
01:05:04,886 --> 01:05:05,086
[ Inaudible Speaker ]

1500
01:05:05,086 --> 01:05:05,656
I'm sorry.

1501
01:05:05,976 --> 01:05:07,446
>> Garbage collection.

1502
01:05:07,516 --> 01:05:09,436
>> Yeah so some kind
of garbage collection

1503
01:05:09,436 --> 01:05:10,946
and which things
would you collect?

1504
01:05:11,046 --> 01:05:13,816
What things would you
remove from memory?

1505
01:05:13,816 --> 01:05:14,266
[ Inaudible Speaker ]

1506
01:05:14,266 --> 01:05:22,996
Good, so we can essentially
expire objects based

1507
01:05:22,996 --> 01:05:24,306
on when they were put in.

1508
01:05:24,306 --> 01:05:27,736
So if I put in user
1, 2, 3 yesterday

1509
01:05:27,736 --> 01:05:30,816
and I haven't touched him
since or needed him since

1510
01:05:30,816 --> 01:05:35,136
and I need more space, well out
goes user 1, 2,3 and I can reuse

1511
01:05:35,136 --> 01:05:38,976
that space, that memory for
user 4, 5, 6 if user 4, 5,

1512
01:05:39,036 --> 01:05:42,306
6 is the next person I'm trying
to insert into the cache.

1513
01:05:42,306 --> 01:05:46,426
So indeed this is a very common
mechanism whereby the first one

1514
01:05:46,426 --> 01:05:48,706
is the first one out if

1515
01:05:48,796 --> 01:05:50,586
that object has not
been needed since.

1516
01:05:51,036 --> 01:05:53,086
By contrast if 1,
2, 3 is just one

1517
01:05:53,086 --> 01:05:56,216
of these power users who's
logging in quite a bit and he --

1518
01:05:56,326 --> 01:05:58,716
he or she is logging and
again, and again, and again,

1519
01:05:58,926 --> 01:06:00,866
well I should remember
every time

1520
01:06:00,866 --> 01:06:02,306
and we don't see it
in the code here.

1521
01:06:02,546 --> 01:06:07,006
But every time I get a cache hit
and I actually find user 1, 2,

1522
01:06:07,056 --> 01:06:10,846
3 in the cache, I could
somehow execute another Memcache

1523
01:06:10,846 --> 01:06:14,026
function that just touches
the user object so to speak.

1524
01:06:14,026 --> 01:06:17,386
Thereby updating his timestamp
to be this moment in time

1525
01:06:17,676 --> 01:06:20,576
so that you remember
that he was just selected

1526
01:06:20,576 --> 01:06:23,026
and hopefully Memcache
get itself would do

1527
01:06:23,026 --> 01:06:24,306
that for us and indeed it does.

1528
01:06:24,306 --> 01:06:25,646
I don't need to this manually,

1529
01:06:25,746 --> 01:06:28,166
the cache software
would remember, Oh,

1530
01:06:28,166 --> 01:06:30,246
you asked for user 1,
2, 3 I should move --

1531
01:06:30,246 --> 01:06:31,896
probably move him back
to the front of the line

1532
01:06:31,896 --> 01:06:34,016
so that the person at the
end of the line is first one

1533
01:06:34,016 --> 01:06:36,426
to get evicted next time around.

1534
01:06:36,866 --> 01:06:38,516
So it's a wonderfully
useful mechanism

1535
01:06:38,516 --> 01:06:40,576
and Facebook is very read heavy

1536
01:06:40,576 --> 01:06:44,156
or very write heavy
if you're a user.

1537
01:06:48,356 --> 01:06:50,566
It's kind of both these
days, you know earlier

1538
01:06:50,566 --> 01:06:53,826
on it was much more read
heavy than write heavy

1539
01:06:53,826 --> 01:06:55,656
because there were no
like status updates

1540
01:06:55,656 --> 01:06:58,246
and you would just have your
profile and that was about it.

1541
01:06:58,436 --> 01:07:00,456
So these days there's definitely
more writes but I'm going

1542
01:07:00,456 --> 01:07:03,966
to guess that reads are still
more common than not, right?

1543
01:07:03,966 --> 01:07:06,306
When you log -- if you're
a Facebook user and you log

1544
01:07:06,306 --> 01:07:09,246
into your accounts and you
see your newsfeed, you know,

1545
01:07:09,246 --> 01:07:11,606
you might have 10, 20
whatever friends show up in

1546
01:07:11,606 --> 01:07:14,076
that newsfeed, that's
potentially like 10

1547
01:07:14,076 --> 01:07:17,996
or 20 queries of some sort and
yet you 're probably not going

1548
01:07:17,996 --> 01:07:21,996
to update your status 30 times
at -- in that same unit of time.

1549
01:07:22,276 --> 01:07:24,686
So odds are Facebook is still
a little more read heavy

1550
01:07:24,876 --> 01:07:27,076
which makes caches is
all the more compelling

1551
01:07:27,286 --> 01:07:30,246
because if your own profile
isn't changing all that often

1552
01:07:30,666 --> 01:07:34,066
at least you might get 10 page
used, 100 page used by friends

1553
01:07:34,066 --> 01:07:36,816
or random strangers before you
actually update your status

1554
01:07:36,816 --> 01:07:37,796
or your profile again.

1555
01:07:38,106 --> 01:07:40,526
That's an opportunity for
optimization, so early on

1556
01:07:40,526 --> 01:07:43,566
and to this day, Facebook uses
things like Memcashe quite a bit

1557
01:07:43,846 --> 01:07:46,716
so that they're not hitting
their various databases just

1558
01:07:46,716 --> 01:07:47,876
to generate your profile.

1559
01:07:48,126 --> 01:07:49,916
They're instead just
getting the results

1560
01:07:50,176 --> 01:07:54,176
of some previous lookup
unless it has since expired.

1561
01:07:54,626 --> 01:07:57,856
Well onto my MySQL optimization

1562
01:07:57,856 --> 01:07:59,786
so that you can squeeze
all the more performance

1563
01:07:59,786 --> 01:08:00,536
out of your set up.

1564
01:08:00,536 --> 01:08:02,636
So this table's a little
more overwhelming right now

1565
01:08:02,636 --> 01:08:04,646
than it needs to be but
recall our discussion

1566
01:08:04,646 --> 01:08:06,886
about MySQL storage
engine some time ago.

1567
01:08:07,186 --> 01:08:09,936
And we talked briefly
about my ISAM and NODB;

1568
01:08:09,936 --> 01:08:12,706
does anyone remember
at least one

1569
01:08:12,706 --> 01:08:14,476
of the distinguishing
characteristics

1570
01:08:14,476 --> 01:08:15,896
of those two storage engines?

1571
01:08:16,076 --> 01:08:18,876
And again the storage engine was
just like the underline format

1572
01:08:19,276 --> 01:08:21,126
that was used to store
your database data.

1573
01:08:22,566 --> 01:08:28,046
>> I think NODB supported
transactions into [inaudible].

1574
01:08:28,046 --> 01:08:30,016
>> Good, so NODB which
is the default these days

1575
01:08:30,066 --> 01:08:31,586
so you haven't really
needed to think

1576
01:08:31,586 --> 01:08:33,386
about this much since
project one.

1577
01:08:33,686 --> 01:08:37,386
NODB supports transactions
whereas MyISAM does not.

1578
01:08:37,386 --> 01:08:41,156
MyISAM uses locks which are
full table locks but does tend

1579
01:08:41,156 --> 01:08:43,446
to have some other
properties and these --

1580
01:08:43,446 --> 01:08:46,166
this list here is
a very long list

1581
01:08:46,296 --> 01:08:49,756
of various distinctions among
these several storage engines.

1582
01:08:49,976 --> 01:08:51,826
Transactions is one of them

1583
01:08:52,166 --> 01:08:54,886
but there's a few other
storage engines here

1584
01:08:54,886 --> 01:08:57,226
that I thought I would
just draw our attention to.

1585
01:08:57,226 --> 01:09:03,556
So one, you have a memory
engine otherwise known

1586
01:09:03,556 --> 01:09:04,726
as a heap engine.

1587
01:09:04,986 --> 01:09:08,026
This is a table that's
intentionally only store in RAM

1588
01:09:08,496 --> 01:09:10,416
which means if you
lose power, server dies

1589
01:09:10,416 --> 01:09:11,726
or what not the entire contents

1590
01:09:11,726 --> 01:09:13,736
of these memory tables
will be lost.

1591
01:09:14,426 --> 01:09:17,746
But still kind of a nice
feature if you yourself want

1592
01:09:17,746 --> 01:09:21,256
to implement a cache relatively
easily by writing keys

1593
01:09:21,256 --> 01:09:23,766
and values into your database
and to two columns maybe,

1594
01:09:24,006 --> 01:09:27,266
you yourself can implement some
kind of cache to avoid having

1595
01:09:27,266 --> 01:09:30,616
to touch maybe much larger
tables that you yourself have.

1596
01:09:30,616 --> 01:09:31,886
So that's an option to you.

1597
01:09:32,246 --> 01:09:36,046
Archive Storage Engine, haven't
had to use this but take a guess

1598
01:09:36,046 --> 01:09:40,106
as to what it does besides
archiving something.

1599
01:09:40,466 --> 01:09:43,956
What does this engine do
for you, do you think?

1600
01:09:52,486 --> 01:09:53,496
>> It doesn't seem to
be doing very much,

1601
01:09:53,496 --> 01:09:55,976
it probably just
stores [inaudible].

1602
01:09:56,066 --> 01:09:57,246
>> And what -- what
was the last sentence,

1603
01:09:57,336 --> 01:09:58,856
the last part of your comment?

1604
01:09:59,196 --> 01:10:02,496
>> It doesn't store anything
in cache [inaudible].

1605
01:10:02,496 --> 01:10:04,326
>> Oh, it doesn't store
in anything in cache,

1606
01:10:04,326 --> 01:10:05,756
you have to query
it all the time.

1607
01:10:05,756 --> 01:10:06,546
Not quite.

1608
01:10:06,686 --> 01:10:10,256
So the property you're actually
getting and you can kind

1609
01:10:10,256 --> 01:10:13,896
of see it here, but
there's some footnotes

1610
01:10:13,896 --> 01:10:16,796
on the other storage engines,
is it's compressed by default.

1611
01:10:17,416 --> 01:10:20,616
So archive tables are
actually slower to query,

1612
01:10:20,966 --> 01:10:23,326
but they're automatically
compressed for you, so they take

1613
01:10:23,326 --> 01:10:24,636
up much, much less space.

1614
01:10:24,796 --> 01:10:27,856
So a common case
-- common use case

1615
01:10:27,896 --> 01:10:31,306
for archive tables might be
log files, where you want

1616
01:10:31,306 --> 01:10:33,836
to keep the data around and you
want to write out a whole bunch

1617
01:10:33,836 --> 01:10:35,926
of key values in a row.

1618
01:10:36,166 --> 01:10:37,626
Every time someone
hits your web server

1619
01:10:37,626 --> 01:10:39,366
or anytime something
buys someone,

1620
01:10:39,796 --> 01:10:42,436
but suppose you rarely query
that data, you're keeping it

1621
01:10:42,436 --> 01:10:44,266
for posterity, for
research purposes,

1622
01:10:44,266 --> 01:10:46,236
for diagnostic purposes,
but you're not going

1623
01:10:46,236 --> 01:10:48,176
to do any selects
on it anytime soon.

1624
01:10:48,386 --> 01:10:50,256
So it would just be
a waste to keep --

1625
01:10:50,376 --> 01:10:52,946
use more disk space than you
need to, so you're willing

1626
01:10:52,946 --> 01:10:55,486
to sacrifice some future
performance when you do need

1627
01:10:55,486 --> 01:10:58,246
to query it for some
long-term disk saving.

1628
01:10:58,286 --> 01:11:00,166
So, the archive format
allows you to do that.

1629
01:11:00,666 --> 01:11:04,476
NDB is a network storage engine,
which is used for clustering.

1630
01:11:04,766 --> 01:11:08,076
So that actually there is a
way of redressing the issue

1631
01:11:08,076 --> 01:11:10,376
of single points of failure
that we discussed earlier

1632
01:11:10,376 --> 01:11:13,356
with shared storage, but
we'll see a simpler approach

1633
01:11:13,486 --> 01:11:14,256
in just a moment.

1634
01:11:15,886 --> 01:11:19,076
So, in the world of
databases, like MySQL,

1635
01:11:19,076 --> 01:11:21,106
they typically offer
this replication feature

1636
01:11:21,406 --> 01:11:22,406
that I mentioned earlier.

1637
01:11:22,656 --> 01:11:25,626
So replication is all about
making automatic copies

1638
01:11:25,626 --> 01:11:28,256
of something and the terminology
generally goes as follows.

1639
01:11:28,256 --> 01:11:30,076
You generally have
a master database,

1640
01:11:30,386 --> 01:11:34,126
which is where you read
data from and write data to,

1641
01:11:34,426 --> 01:11:37,216
but just for good measure,
that master has one

1642
01:11:37,216 --> 01:11:40,716
or more slave databases attached
to it by a network connection,

1643
01:11:40,996 --> 01:11:45,686
and their purpose in life is
to get a copy of row that's

1644
01:11:45,686 --> 01:11:46,966
in the master database.

1645
01:11:46,966 --> 01:11:48,266
You can think of
it rather simply

1646
01:11:48,266 --> 01:11:50,926
as anytime a query is
executed on the master,

1647
01:11:51,136 --> 01:11:54,036
that same query is copied
down to one or more slaves

1648
01:11:54,036 --> 01:11:55,446
and they do the exact
same thing.

1649
01:11:55,616 --> 01:11:58,546
So that in theory, the master
in all of these slaves,

1650
01:11:58,546 --> 01:12:00,336
are identical to one another.

1651
01:12:00,936 --> 01:12:05,126
So what's the upside now of
having databases one, two,

1652
01:12:05,126 --> 01:12:06,876
three and four, all
of which are copies

1653
01:12:06,876 --> 01:12:07,926
of one another apparently?

1654
01:12:08,736 --> 01:12:12,346
What problems does this
solve for us, if any?

1655
01:12:12,346 --> 01:12:12,616
Axle?

1656
01:12:12,966 --> 01:12:19,696
>> Well, database one dies
because somebody trips

1657
01:12:19,736 --> 01:12:23,286
over the cord, but you
still have three copies.

1658
01:12:23,286 --> 01:12:24,406
>> Good, so if something --

1659
01:12:24,406 --> 01:12:28,236
if database one dies
because of human error,

1660
01:12:28,236 --> 01:12:32,146
you trip over the cord, hard
drive dies, RAM fizzles out,

1661
01:12:32,146 --> 01:12:34,896
whatever the case may be, you
have literally three backups

1662
01:12:34,926 --> 01:12:38,616
that are identical, so
there's no tapes involved,

1663
01:12:38,746 --> 01:12:39,746
there's no backup server.

1664
01:12:39,746 --> 01:12:41,826
I mean, these are
full-fledged databases

1665
01:12:41,826 --> 01:12:45,106
and in the simplest case, you
could just unplug the master,

1666
01:12:45,166 --> 01:12:46,926
plug in the slave and viola,
you now have a new master.

1667
01:12:47,016 --> 01:12:49,596
And you might have to do
a bit of reconfiguration

1668
01:12:49,696 --> 01:12:54,036
in the databases to make --
to promote him to master,

1669
01:12:54,036 --> 01:12:57,206
so to speak and then leave
servers three and four

1670
01:12:57,206 --> 01:12:59,816
as the new slaves while
you fix server number one.

1671
01:12:59,856 --> 01:13:01,276
But that would be
one approach here,

1672
01:13:01,276 --> 01:13:02,566
so you have some redundancy,

1673
01:13:02,566 --> 01:13:04,396
even though you might have
a little bit of downtime,

1674
01:13:04,396 --> 01:13:05,686
at least you can get back
up and running quickly.

1675
01:13:05,686 --> 01:13:07,086
And indeed, you can automate
this process if you notice

1676
01:13:07,116 --> 01:13:07,686
that the master is down,

1677
01:13:07,716 --> 01:13:09,096
you could take him offline
completely, promote the slave

1678
01:13:09,126 --> 01:13:10,446
and reconfigure them all
just by writing a script.

1679
01:13:10,476 --> 01:13:11,976
What else -- how else can we
take advantage of this topology?

1680
01:13:21,076 --> 01:13:23,126
Let me ask a more
leading question.

1681
01:13:24,026 --> 01:13:27,116
In the context of Facebook,
especially early days,

1682
01:13:27,996 --> 01:13:31,066
how might they in
particular have made good use

1683
01:13:31,066 --> 01:13:32,036
of this topology?

1684
01:13:32,536 --> 01:13:32,706
Axle?

1685
01:13:37,556 --> 01:13:42,166
>> Well, maybe if you get a
lot of the -- a lot of queries,

1686
01:13:42,166 --> 01:13:45,296
you can outsource them
to different slaves.

1687
01:13:45,446 --> 01:13:47,656
>> Okay, so if you're getting
a lot of queries, you could,

1688
01:13:47,656 --> 01:13:49,206
you know, maybe you
could just load balance

1689
01:13:49,366 --> 01:13:52,226
across database servers,
and absolutely you could.

1690
01:13:52,226 --> 01:13:54,536
The load balancers don't have
to be used for HDP alone.

1691
01:13:54,866 --> 01:13:56,356
You could use it
for MySQL traffic,

1692
01:13:56,356 --> 01:13:58,496
but why do I say
Facebook in particular?

1693
01:13:59,096 --> 01:14:00,966
Early on they didn't get
that many queries, but it --

1694
01:14:00,966 --> 01:14:05,776
this was still a good
paradigm for them.

1695
01:14:05,776 --> 01:14:06,316
[ Inaudible Speaker ]

1696
01:14:06,316 --> 01:14:08,646
Yeah, why is this good, perhaps?

1697
01:14:16,156 --> 01:14:18,416
So, back to my hypothesis

1698
01:14:18,446 --> 01:14:20,886
that they're more read
heavy than write heavy.

1699
01:14:21,516 --> 01:14:25,046
How can you adapt that --

1700
01:14:25,956 --> 01:14:29,856
that reality to this
particular topology effectively?

1701
01:14:30,556 --> 01:14:34,396
Or put another way, why is this
a good topology for a website

1702
01:14:34,446 --> 01:14:36,896
that is very read heavy
and less write heavy?

1703
01:14:37,786 --> 01:14:37,946
Ben?

1704
01:14:38,516 --> 01:14:44,356
[ Inaudible Speaker ]

1705
01:14:44,856 --> 01:14:45,086
Okay.

1706
01:14:45,346 --> 01:14:45,986
>> But if you're not
writing [inaudible].

1707
01:14:48,246 --> 01:14:48,856
>> Okay, good.

1708
01:14:48,856 --> 01:14:51,476
So reading can be expedited,
so if we kind of combine Ben

1709
01:14:51,476 --> 01:14:54,436
and Axles' proposals here,
for a read heavy website

1710
01:14:54,436 --> 01:14:56,276
like Facebook, certainly
in the early days,

1711
01:14:56,506 --> 01:14:58,156
you could just write
your code in such a way

1712
01:14:58,156 --> 01:15:02,716
that any select statements go
to databases two, three or four

1713
01:15:02,716 --> 01:15:05,896
and any inserts, updates or
deletes have to go apparently

1714
01:15:05,966 --> 01:15:11,156
to server one, which even though
that query than has to propagate

1715
01:15:11,156 --> 01:15:13,336
to servers two, three and
four, it is less common.

1716
01:15:13,596 --> 01:15:16,456
And that happens automatically,
so the code wise, you don't have

1717
01:15:16,456 --> 01:15:18,786
to worry about it too much
and if you're suffering a bit

1718
01:15:18,786 --> 01:15:20,846
of performance there, well you
can just throw more servers

1719
01:15:20,846 --> 01:15:22,366
at it and have even
more read servers

1720
01:15:22,406 --> 01:15:23,636
to lighten the load further.

1721
01:15:23,896 --> 01:15:25,946
So this approach
of having slaves

1722
01:15:25,946 --> 01:15:28,306
that can typically be used
either for redundancy,

1723
01:15:28,306 --> 01:15:31,916
so you just have a hot
spare ready to go or so

1724
01:15:31,916 --> 01:15:34,136
that you can balance
read requests

1725
01:15:34,696 --> 01:15:38,406
across them is a very nice
solution, but of course,

1726
01:15:39,036 --> 01:15:41,676
every time we solve one problem,
we've introduced another

1727
01:15:41,676 --> 01:15:43,726
or at least we haven't
fixed yet another here.

1728
01:15:44,376 --> 01:15:48,586
What is a -- what is a
fault in this layout still?

1729
01:15:48,996 --> 01:15:51,136
Be paranoid.

1730
01:15:55,856 --> 01:15:57,416
Kind of talked about it earlier,

1731
01:15:57,416 --> 01:16:00,776
but like what if
one dies, right?

1732
01:16:00,776 --> 01:16:04,656
Unless this is - there's got to
be some blip on the radar here

1733
01:16:04,726 --> 01:16:07,926
because we have to like
promote a slave and, right so,

1734
01:16:07,926 --> 01:16:09,156
you still have a single point

1735
01:16:09,156 --> 01:16:10,666
of failure here, at
least for writes.

1736
01:16:10,856 --> 01:16:14,676
We could keep Facebook alive by
letting people browse profiles

1737
01:16:14,676 --> 01:16:16,686
and read profiles, but
status updates for instance,

1738
01:16:16,686 --> 01:16:18,646
could be offline for
as long as it takes us

1739
01:16:18,646 --> 01:16:20,736
to promote a slave to a master.

1740
01:16:21,096 --> 01:16:22,446
Feels like it'd be nicer

1741
01:16:22,446 --> 01:16:24,646
or at least our probability
would be better of uptime

1742
01:16:24,926 --> 01:16:28,386
if we instead had not just
a single master, but again,

1743
01:16:28,386 --> 01:16:29,786
let's just throw
hardware at the problem.

1744
01:16:29,786 --> 01:16:31,876
So another common
paradigm is actually

1745
01:16:31,876 --> 01:16:36,656
to have a master-master setup
whereby as the labels imply

1746
01:16:36,716 --> 01:16:39,556
and as the arrows suggest,
this time you could write

1747
01:16:39,686 --> 01:16:43,156
to either server one or two,
and if you happen to write

1748
01:16:43,156 --> 01:16:45,606
to server one, that
query gets replicated

1749
01:16:45,606 --> 01:16:47,796
on server two and vice versa.

1750
01:16:48,336 --> 01:16:50,086
So now, you could
keep it simple.

1751
01:16:50,086 --> 01:16:53,936
You could always write to
one, but then the query goes

1752
01:16:53,936 --> 01:16:57,446
to number two automatically
or you could write to either,

1753
01:16:57,446 --> 01:16:59,246
thereby load balancing
across the two

1754
01:16:59,246 --> 01:17:01,896
and they'll propagate
between each other.

1755
01:17:02,126 --> 01:17:04,566
But in this case
here, if you've laid

1756
01:17:04,566 --> 01:17:07,576
out your network connections
properly, either one

1757
01:17:07,576 --> 01:17:10,196
or two can go down, and
you still have a master

1758
01:17:10,476 --> 01:17:11,496
that you could read from,

1759
01:17:11,496 --> 01:17:12,986
and you could even
implement this in code.

1760
01:17:12,986 --> 01:17:15,996
Recall very simply, we had the
MySQL connect function weeks

1761
01:17:15,996 --> 01:17:18,726
ago, or even the PDO
constructor function,

1762
01:17:18,916 --> 01:17:20,446
which tries to connect
to a database.

1763
01:17:20,726 --> 01:17:24,486
You can implement this in PHP
code if MySQL connect fails

1764
01:17:24,486 --> 01:17:27,366
when connecting to server
one, then just try server two.

1765
01:17:27,536 --> 01:17:29,696
So you yourself could
build in some redundancy

1766
01:17:29,696 --> 01:17:32,836
so that now we can lose
server one or two and not have

1767
01:17:32,876 --> 01:17:34,696
to intervene as humans
just yet because we

1768
01:17:34,696 --> 01:17:36,756
at least still have
a second master

1769
01:17:36,756 --> 01:17:38,106
that we can continue writing

1770
01:17:38,106 --> 01:17:43,056
to even though server one is now
offline for some amount of time.

1771
01:17:43,816 --> 01:17:46,096
All right, but we still
have to route traffic there,

1772
01:17:46,096 --> 01:17:48,826
so in pursuit of this
idea of load balancing,

1773
01:17:48,826 --> 01:17:51,876
here's a more complex picture
that starts to unite some

1774
01:17:51,876 --> 01:17:54,646
of our web ideas and some
of our database ideas.

1775
01:17:54,646 --> 01:17:56,506
So at the top there we
have some kind of network.

1776
01:17:56,866 --> 01:17:58,736
We have a load balancer
in between

1777
01:17:58,936 --> 01:18:00,906
and then we have
this front-end tier.

1778
01:18:00,906 --> 01:18:04,626
So web servers are typically
called a tier, a service tier,

1779
01:18:04,626 --> 01:18:06,816
this is a multi-tiered
architecture,

1780
01:18:06,816 --> 01:18:07,856
would be the jargon here.

1781
01:18:08,476 --> 01:18:11,506
And those web servers now
apparently are routing their

1782
01:18:11,506 --> 01:18:13,836
requests through what in order

1783
01:18:13,836 --> 01:18:16,256
to reach some MySQL
slaves for reads?

1784
01:18:20,556 --> 01:18:22,046
Who's the man in
the middle here?

1785
01:18:22,126 --> 01:18:22,646
Yeah, Axle?

1786
01:18:23,096 --> 01:18:26,546
>> For reads with
a load balancer.

1787
01:18:26,636 --> 01:18:27,916
>> Okay, so for reads, yeah,

1788
01:18:27,946 --> 01:18:30,756
we have a second load
balancer depicted here,

1789
01:18:31,096 --> 01:18:33,086
frankly in reality they
could be one in the same.

1790
01:18:33,086 --> 01:18:35,166
They could be the same
device, but just listening

1791
01:18:35,166 --> 01:18:36,436
for different types
of connections.

1792
01:18:36,816 --> 01:18:38,896
But for now, they're drawn
more simply as separate.

1793
01:18:39,136 --> 01:18:42,266
Now we have one MySQL
master, so we also have wires

1794
01:18:42,266 --> 01:18:45,246
or arrows pointing from the
web servers to the master.

1795
01:18:45,246 --> 01:18:46,646
And the master meanwhile,
has some kind

1796
01:18:46,646 --> 01:18:47,856
of connection to the slaves.

1797
01:18:48,336 --> 01:18:51,646
So not bad, no frankly, this
is starting to hurt my brain

1798
01:18:51,646 --> 01:18:54,066
because now what was
a very simple class

1799
01:18:54,066 --> 01:18:56,706
where you have a nice
self contained appliance

1800
01:18:56,706 --> 01:18:59,176
on your machine -- on your
laptop, does everything, web,

1801
01:18:59,526 --> 01:19:01,716
database, caching,
anything you want it to do.

1802
01:19:02,056 --> 01:19:04,286
My God, look at all the
things we have to wire up now

1803
01:19:04,286 --> 01:19:05,246
and it's still now perfect.

1804
01:19:05,556 --> 01:19:06,476
What could die here?

1805
01:19:06,476 --> 01:19:08,166
Where -- what are our
single points of failure?

1806
01:19:09,426 --> 01:19:09,916
Jack?

1807
01:19:09,916 --> 01:19:10,846
[ Inaudible Speaker ]

1808
01:19:10,846 --> 01:19:11,646
Oh, Isaac, sure.

1809
01:19:13,316 --> 01:19:15,086
>> All right, MySQL master.

1810
01:19:15,586 --> 01:19:16,906
>> Okay, so the MySQL master.

1811
01:19:16,906 --> 01:19:19,566
We haven't really solved that
problem, so kind of be nice

1812
01:19:19,616 --> 01:19:21,176
to steal part of
the previous picture

1813
01:19:21,176 --> 01:19:23,656
and maybe insert it into here.

1814
01:19:23,816 --> 01:19:24,106
Jack?

1815
01:19:24,106 --> 01:19:25,496
>> Oh, I was [inaudible].

1816
01:19:26,046 --> 01:19:26,396
>> Same thing.

1817
01:19:26,536 --> 01:19:26,906
Axle?

1818
01:19:27,766 --> 01:19:28,516
>> [Inaudible] load balancer.

1819
01:19:29,096 --> 01:19:29,956
>> Load balancers, right.

1820
01:19:29,956 --> 01:19:33,136
So single point of failure
is very well defined,

1821
01:19:33,136 --> 01:19:36,056
like single point of
failure, just look for any --

1822
01:19:37,276 --> 01:19:41,276
any bottlenecks here, whereby
things point in and then go out.

1823
01:19:41,496 --> 01:19:44,316
Load balancer is one here,
load balancer is one there,

1824
01:19:44,516 --> 01:19:47,496
so it turns out with load
balancers for your $100,000,

1825
01:19:47,496 --> 01:19:49,686
you can get two of them
typically in the package.

1826
01:19:49,956 --> 01:19:53,156
And what they tend to do is
operate also on what's called,

1827
01:19:53,826 --> 01:19:55,626
similar in spirit to
master-master mode,

1828
01:19:55,936 --> 01:19:57,376
but in the context
of load balancers,

1829
01:19:57,376 --> 01:19:59,486
it's typically called
active-active,

1830
01:19:59,696 --> 01:20:00,976
as opposed to active-passive.

1831
01:20:01,426 --> 01:20:04,816
And the idea here is with
active-active, you have a pair

1832
01:20:04,816 --> 01:20:07,226
of load balancers that
are constantly listening

1833
01:20:07,306 --> 01:20:08,196
for connections.

1834
01:20:08,536 --> 01:20:13,246
Either one of which can receive
packets from the outside world

1835
01:20:13,246 --> 01:20:16,266
and then relay them
to backend servers.

1836
01:20:16,436 --> 01:20:19,136
And what they typically do is
they send heartbeats from left

1837
01:20:19,136 --> 01:20:21,176
to right and right
to left so that

1838
01:20:21,176 --> 01:20:24,216
if this guy ever stops hearing
a heart beat from this guy,

1839
01:20:24,216 --> 01:20:26,036
so to speak and a heartbeat
is just like a packet

1840
01:20:26,276 --> 01:20:28,246
that gets sent every second
or something like that.

1841
01:20:28,556 --> 01:20:31,596
If this guy stops hearing
a heartbeat from this guy,

1842
01:20:31,866 --> 01:20:34,586
he automatically assumes that
this guy must have gone offline

1843
01:20:34,586 --> 01:20:38,506
so he's completely in charge now
and he continues to send traffic

1844
01:20:38,706 --> 01:20:41,066
from the outside world in,

1845
01:20:41,376 --> 01:20:43,366
or if you instead have
active-passive mode,

1846
01:20:43,616 --> 01:20:45,566
if this is the active
guy at the moment,

1847
01:20:45,966 --> 01:20:49,286
rather if this is the active
guy at the moment and he dies,

1848
01:20:49,586 --> 01:20:51,946
this guy similarly detects,
"Ooh, no more heartbeat."

1849
01:20:52,066 --> 01:20:55,246
And what the passive guy will
do is promote himself to active,

1850
01:20:55,246 --> 01:20:56,696
which essentially
just means he takes

1851
01:20:56,696 --> 01:20:58,316
over the other guys IP address

1852
01:20:58,806 --> 01:21:00,676
so that all traffic
now comes to him.

1853
01:21:00,986 --> 01:21:03,806
So in short, we definitely
need another load balancer

1854
01:21:03,806 --> 01:21:06,016
in the picture, how
it's implemented is --

1855
01:21:06,946 --> 01:21:09,696
is not as important
to us right now,

1856
01:21:10,056 --> 01:21:12,556
but having a single load
balancer is probably a bad

1857
01:21:12,786 --> 01:21:13,176
thing, right.

1858
01:21:13,176 --> 01:21:15,616
And this is the tragedy,
you can throw money at,

1859
01:21:15,616 --> 01:21:18,266
you can throw a lot of brain
power at various tiers here,

1860
01:21:18,476 --> 01:21:21,526
but if you have a lot of web
servers, a lot of MySQL servers,

1861
01:21:21,926 --> 01:21:23,546
but you have one
load balancer just

1862
01:21:23,546 --> 01:21:25,156
because it was really
expensive or you didn't know how

1863
01:21:25,156 --> 01:21:28,756
to configure it properly, like
the rest of it is pretty much

1864
01:21:28,756 --> 01:21:32,186
for naught because you still
have things that can die

1865
01:21:32,506 --> 01:21:34,756
and take down your
entire website.

1866
01:21:35,096 --> 01:21:36,646
So let's make this
more complex still.

1867
01:21:36,946 --> 01:21:40,306
So let's now introduce
two load balancers

1868
01:21:40,476 --> 01:21:43,296
and let's actually introduce
an idea of partitioning.

1869
01:21:43,296 --> 01:21:44,336
And this was actually something

1870
01:21:44,496 --> 01:21:47,206
that Facebook coincidentally
did make good use of early on.

1871
01:21:47,526 --> 01:21:49,666
Back in the day, there
was Harvard.facebook.

1872
01:21:49,666 --> 01:21:53,736
-- Harvard.thefacebook.com,
there was mit.thefacebook.com

1873
01:21:54,066 --> 01:21:56,526
and the earliest
partitioning that they used was

1874
01:21:56,526 --> 01:21:58,676
to essentially have
a different server,

1875
01:21:58,676 --> 01:22:01,466
as best outsiders could
tell for each school.

1876
01:22:01,466 --> 01:22:03,326
So they literally just
like copied the database,

1877
01:22:03,356 --> 01:22:06,906
copied the files over to
another server and then viola,

1878
01:22:06,906 --> 01:22:10,136
thus was born MIT's
copy of Facebook.

1879
01:22:10,366 --> 01:22:12,806
But this is actually, even
though this would get kind

1880
01:22:12,806 --> 01:22:15,766
of messy for 800 million users
and thousands and thousands

1881
01:22:15,766 --> 01:22:19,536
of universities and networks,
it's pretty clean early

1882
01:22:19,536 --> 01:22:23,276
on because it's just to leverage
this idea of partitioning,

1883
01:22:23,276 --> 01:22:24,676
right, it's kind of
-- we didn't have a --

1884
01:22:24,676 --> 01:22:27,316
Facebook didn't have a big
enough server to handle Harvard

1885
01:22:27,316 --> 01:22:31,346
and MIT, so why not just get two
and say Harvard users go here,

1886
01:22:31,346 --> 01:22:35,096
MIT users go here and now we've
kind of avoided that problem.

1887
01:22:35,126 --> 01:22:37,796
Now unfortunately, when BU comes
on, we need a third server,

1888
01:22:38,056 --> 01:22:40,066
but at least we can
scale horizontally.

1889
01:22:40,386 --> 01:22:42,046
Now there is a catch
with partitioning,

1890
01:22:42,326 --> 01:22:44,856
as soon as you wanted to be
able to poke someone at MIT

1891
01:22:44,856 --> 01:22:47,226
or vice versa, you
had to somehow cross

1892
01:22:47,366 --> 01:22:50,466
that Harvard/MIT boundary
at which point it's kind

1893
01:22:50,466 --> 01:22:52,706
of a bad thing that they're
all in separate databases.

1894
01:22:52,706 --> 01:22:55,156
So early on, there were some
features that you could only do

1895
01:22:55,366 --> 01:22:56,786
within your own network,

1896
01:22:56,786 --> 01:22:59,646
not until there was more share
state could you send messages

1897
01:22:59,646 --> 01:23:00,176
and the like.

1898
01:23:00,626 --> 01:23:03,586
So partitioning though could
be used even more simply.

1899
01:23:03,926 --> 01:23:06,006
Suppose that you just had
a whole bunch of users,

1900
01:23:06,726 --> 01:23:09,966
well you need to scale your
architecture horizontally,

1901
01:23:10,176 --> 01:23:12,826
why don't I just put users who's
last names start with a to m

1902
01:23:13,156 --> 01:23:17,406
on half of my servers and then n
through z on the others, right.

1903
01:23:17,406 --> 01:23:19,016
And when they log in,
I just send them to one

1904
01:23:19,016 --> 01:23:20,596
or the other server
based on that.

1905
01:23:20,776 --> 01:23:23,446
So in general, partitioning
is not such a bad idea,

1906
01:23:23,746 --> 01:23:25,766
it's very common in databases

1907
01:23:25,766 --> 01:23:27,596
because you can still
have redundancy,

1908
01:23:27,596 --> 01:23:30,236
whole bunch of slaves in this
case here, whole bunch of slaves

1909
01:23:30,236 --> 01:23:32,696
over here, but you
can balance load based

1910
01:23:32,696 --> 01:23:35,766
on some high level user
information, not based on load,

1911
01:23:35,766 --> 01:23:37,726
not round robin, you
can actually take

1912
01:23:37,726 --> 01:23:40,406
into account what someone's
name is and then send them

1913
01:23:40,406 --> 01:23:41,876
to this particular server.

1914
01:23:42,366 --> 01:23:44,556
So partitioning is a
very common paradigm.

1915
01:23:44,946 --> 01:23:47,636
And then lastly just
to slap a word on it,

1916
01:23:47,636 --> 01:23:51,246
high availability refers to
what we described in the context

1917
01:23:51,246 --> 01:23:54,416
of load balancers, but it can
apply to databases as well.

1918
01:23:54,416 --> 01:23:58,826
Whereby high availability or HA
is the buzzword, simply refers

1919
01:23:58,826 --> 01:24:01,446
to some kind of relationship
between a pair or more

1920
01:24:01,446 --> 01:24:04,546
of servers that are somehow
checking each other's

1921
01:24:04,546 --> 01:24:07,456
heartbeats, so that if one
of them dies, the other takes

1922
01:24:07,456 --> 01:24:09,696
on the entire burden of the --

1923
01:24:10,176 --> 01:24:12,926
of the service that's being
provided, whether a database

1924
01:24:13,406 --> 01:24:15,306
or whether a load balancer.

1925
01:24:15,306 --> 01:24:19,956
So, even though we finally
got the iPad working,

1926
01:24:19,956 --> 01:24:22,956
it's a little small to draw
on, so what I wanted to do

1927
01:24:23,356 --> 01:24:25,666
as our final example here,

1928
01:24:25,666 --> 01:24:30,666
is let me raise the
screen here [inaudible] one

1929
01:24:31,016 --> 01:24:34,336
of these buttons will do it.

1930
01:24:34,336 --> 01:24:39,086
All right, we're
going old school now.

1931
01:24:39,216 --> 01:24:46,496
Our first and last piece of
chalk in the class, let's see.

1932
01:24:48,166 --> 01:24:51,006
We'll start with the middle one.

1933
01:24:51,006 --> 01:24:56,396
All right, let's build
ourselves a network here.

1934
01:24:56,946 --> 01:25:01,566
So, we have a need for
one or more web servers,

1935
01:25:01,956 --> 01:25:04,626
one or more databases,
maybe some load balancers,

1936
01:25:04,626 --> 01:25:05,526
but we're also going to try

1937
01:25:05,526 --> 01:25:07,916
to tie together last week's
conversation about security,

1938
01:25:08,206 --> 01:25:10,776
so we'll have to think about
firewalling things out now.

1939
01:25:11,246 --> 01:25:16,726
So in very simple form,
we have here a web server,

1940
01:25:16,826 --> 01:25:17,896
which I'll draw as www.

1941
01:25:17,896 --> 01:25:20,486
All right, so that's
our web server.

1942
01:25:20,486 --> 01:25:22,266
And now my website's
doing so well

1943
01:25:22,266 --> 01:25:24,666
that I need a second web
server, so I'm going to draw it

1944
01:25:24,666 --> 01:25:27,166
like this and now we need

1945
01:25:27,166 --> 01:25:29,576
to revisit the issue
of balancing load.

1946
01:25:30,056 --> 01:25:32,676
So what felt like one of
our best options here?

1947
01:25:33,626 --> 01:25:37,976
How do I still have the
internet, which I'll draw

1948
01:25:37,976 --> 01:25:43,156
as a cloud here, connected to
both of these servers somehow,

1949
01:25:43,716 --> 01:25:45,756
but I want the property
of sticky sessions.

1950
01:25:46,436 --> 01:25:52,886
So what are my options or
what was my best option?

1951
01:25:53,016 --> 01:25:54,866
How do I implement
sticky sessions?

1952
01:25:55,506 --> 01:25:55,676
Axle?

1953
01:25:57,216 --> 01:26:01,116
>> Use a load balancer and keep
all the sessions [inaudible].

1954
01:26:02,056 --> 01:26:02,616
>> Okay, good.

1955
01:26:02,616 --> 01:26:05,616
So use a load balancer and store
all the sessions in one place.

1956
01:26:05,876 --> 01:26:09,016
And that's not -- okay,
so we can actually do one,

1957
01:26:09,016 --> 01:26:10,696
but not necessarily
both of those.

1958
01:26:10,696 --> 01:26:14,466
Let me interpose now the thing
we started calling black box.

1959
01:26:14,936 --> 01:26:16,696
So this is some kind
load balancer,

1960
01:26:16,696 --> 01:26:19,036
now I still have
my backend servers.

1961
01:26:19,546 --> 01:26:21,276
And here's another one here.

1962
01:26:22,356 --> 01:26:24,706
Okay. And now this
is connected here,

1963
01:26:24,846 --> 01:26:27,366
but I still want sticky
sessions, but you know what?

1964
01:26:27,366 --> 01:26:29,796
Shared states, that
sounded expensive,

1965
01:26:29,796 --> 01:26:31,946
fiber channel [inaudible]
sounded complicated,

1966
01:26:32,106 --> 01:26:33,056
there's simpler way.

1967
01:26:33,056 --> 01:26:35,726
How do I ensure that I get
sticky sessions using only a

1968
01:26:35,726 --> 01:26:37,576
load balancer and
no shared state yet?

1969
01:26:45,046 --> 01:26:49,236
How can I ensure that when
Alice comes in and she's sent

1970
01:26:49,236 --> 01:26:52,016
to this server the first time,
that the next time she comes in,

1971
01:26:52,016 --> 01:26:53,596
she's sent to the same one?

1972
01:26:54,206 --> 01:26:54,486
Axle?

1973
01:26:54,566 --> 01:27:00,496
>> Remember which server she
visits either for [inaudible].

1974
01:27:00,716 --> 01:27:01,376
>> Okay, good.

1975
01:27:01,376 --> 01:27:05,416
So why don't we have the load
balancer listen at the HDP level

1976
01:27:05,416 --> 01:27:08,776
and when the response comes
back from the first web server,

1977
01:27:08,776 --> 01:27:09,696
let's give them numbers.

1978
01:27:09,696 --> 01:27:12,056
So let's call this one,
and this guy two --

1979
01:27:12,346 --> 01:27:14,796
the load balancer can
insert some kind of cookie

1980
01:27:15,056 --> 01:27:19,786
that allows it to remember that
this user belongs on server one.

1981
01:27:19,786 --> 01:27:21,896
And how it does that, I don't
know, it's a big random number

1982
01:27:21,896 --> 01:27:25,446
and it's got a table like
PHP does for its sessions

1983
01:27:25,716 --> 01:27:28,966
and figures out which server
to send her to in this case.

1984
01:27:29,316 --> 01:27:30,726
All right, so now
I have a database.

1985
01:27:30,726 --> 01:27:34,336
The easiest way I know how to
set up a database is to put it

1986
01:27:34,336 --> 01:27:35,516
on the web server itself.

1987
01:27:35,586 --> 01:27:37,616
So much like the CS50
appliance, you have a database

1988
01:27:37,616 --> 01:27:40,116
in the same box as a web server.

1989
01:27:40,566 --> 01:27:44,836
If I now have a database here
and here on the same boxes

1990
01:27:44,836 --> 01:27:48,236
as our web servers, what's
the most obvious problem

1991
01:27:48,236 --> 01:27:49,006
that now arises?

1992
01:27:49,566 --> 01:27:49,656
Yeah?

1993
01:27:50,816 --> 01:27:53,336
>> Well, Alice's
shopping cart is --

1994
01:27:53,446 --> 01:27:58,266
oh, well if she does something
to her profile or whatever --

1995
01:27:58,756 --> 01:27:58,996
>> Good.

1996
01:27:59,046 --> 01:28:05,386
>> -- it's just going to be on
server one and not on server two

1997
01:28:05,386 --> 01:28:06,736
since she just visited
server one.

1998
01:28:06,736 --> 01:28:07,246
>> Exactly.

1999
01:28:07,246 --> 01:28:10,496
So if Alice just happens
to end up on server one

2000
01:28:10,566 --> 01:28:13,396
and she updates her profile,
her credit card information

2001
01:28:13,426 --> 01:28:14,596
or something persistent.

2002
01:28:14,866 --> 01:28:16,366
Not the shopping cart thing

2003
01:28:16,366 --> 01:28:18,776
because that involves
the session,

2004
01:28:18,776 --> 01:28:20,976
but she does something
persistent,

2005
01:28:20,976 --> 01:28:25,616
it's going to persist on
this database and that's fine

2006
01:28:25,616 --> 01:28:29,056
because sticky sessions are
solving all my problems now.

2007
01:28:29,166 --> 01:28:35,196
But she comes back
in a week or she logs

2008
01:28:35,196 --> 01:28:39,916
in from a different computer,
cookie expires, whatever,

2009
01:28:40,096 --> 01:28:42,646
and she ends up here --

2010
01:28:42,646 --> 01:28:44,526
what happened to my
credit card information?

2011
01:28:44,526 --> 01:28:45,796
What happened to my profile?

2012
01:28:45,796 --> 01:28:50,186
I now have no profile because
I'm on a different database.

2013
01:28:50,186 --> 01:28:51,866
So clearly, this is not going

2014
01:28:51,866 --> 01:28:53,976
to fly unless we
partition our users

2015
01:28:54,426 --> 01:29:00,006
and have the load balancer
actually take into account

2016
01:29:00,006 --> 01:29:08,436
who is this user and
then send the user based

2017
01:29:08,436 --> 01:29:12,696
on Alice's last name
always to the same servers,

2018
01:29:12,736 --> 01:29:14,996
so that could be one approach.

2019
01:29:15,036 --> 01:29:18,236
But for now, let's instead
factor out the database and say

2020
01:29:18,236 --> 01:29:20,886
that it's not on the web
servers, it's separate

2021
01:29:20,886 --> 01:29:24,306
and it's got some kind of
internet connection here.

2022
01:29:25,296 --> 01:29:27,496
Of course we've solved
one problem,

2023
01:29:27,496 --> 01:29:29,196
but introduced a new
one, which is what?

2024
01:29:29,196 --> 01:29:29,346
Isaac?

2025
01:29:29,346 --> 01:29:30,446
>> Single point of
failure [inaudible].

2026
01:29:30,446 --> 01:29:38,836
>> Yeah. So single
point of failure again.

2027
01:29:38,836 --> 01:29:42,326
So how can we mitigate this?

2028
01:29:42,526 --> 01:29:43,656
Well, we can do a
couple of things.

2029
01:29:43,656 --> 01:29:45,846
We can attach slave
databases off of this,

2030
01:29:45,846 --> 01:29:46,836
and that's kind of nice.

2031
01:29:46,866 --> 01:29:48,066
But it then involves
somehow promoting a slave

2032
01:29:48,096 --> 01:29:48,966
to a master in the
even one dies.

2033
01:29:48,996 --> 01:29:50,136
So maybe the cleanest
approach would be something

2034
01:29:50,166 --> 01:29:50,826
like two master databases,

2035
01:29:50,856 --> 01:29:51,786
so we'll call this
db1, this will be db2.

2036
01:29:51,816 --> 01:29:53,256
And now, how do I want to do
this, connect these like this.

2037
01:29:53,286 --> 01:29:53,856
Isaac, you shook your head.

2038
01:29:53,886 --> 01:29:55,176
>> Well, you could connect the
two databases to each other.

2039
01:29:55,206 --> 01:29:56,076
>> Okay, so we should
probably do this

2040
01:29:56,106 --> 01:29:56,886
for master-master replication.

2041
01:29:56,916 --> 01:29:57,246
So that's good.

2042
01:29:57,276 --> 01:29:58,146
But what about these
lines -- good, bad?

2043
01:29:58,176 --> 01:29:58,506
Answer is bad.

2044
01:29:58,536 --> 01:29:58,836
Why bad, Jack?

2045
01:29:58,866 --> 01:29:59,976
>> Well, you can connect each
one to both of the [inaudible].

2046
01:30:00,296 --> 01:30:03,536
>> Right, so the problem we
just identified was the database

2047
01:30:03,536 --> 01:30:06,646
on same server is web server bad
because then it's talking only

2048
01:30:06,646 --> 01:30:08,856
to it and if Alice ends
up on the other servers,

2049
01:30:08,856 --> 01:30:10,446
she has no data that
you're expecting.

2050
01:30:10,696 --> 01:30:12,126
Well, functionally
this is equivalent.

2051
01:30:12,126 --> 01:30:14,276
I've just drawn a line, but
they're still only connected

2052
01:30:14,276 --> 01:30:17,526
to each other and the
traffic should probably not go

2053
01:30:17,526 --> 01:30:19,136
like that, so we at least need

2054
01:30:19,136 --> 01:30:20,456
to have some kind
of cross connect.

2055
01:30:20,906 --> 01:30:26,526
So okay, so I can do this,
but now what do I do?

2056
01:30:26,526 --> 01:30:28,836
So now my load balancing
has to be done in code,

2057
01:30:28,956 --> 01:30:32,876
if those are the only components
in my systems right now,

2058
01:30:32,876 --> 01:30:35,686
the line suggests that dub, dub,
dub one has a network connection

2059
01:30:35,686 --> 01:30:39,186
to db1 and db2, but that means
now I have to do something

2060
01:30:39,186 --> 01:30:41,956
like an if condition only
in my PHP code to say

2061
01:30:41,956 --> 01:30:44,116
if this database
is up right here,

2062
01:30:44,246 --> 01:30:45,946
else if this database
is up right there.

2063
01:30:45,946 --> 01:30:48,146
And that's not bad, but
now you're developers have

2064
01:30:48,146 --> 01:30:49,846
to know something
about the topology.

2065
01:30:50,036 --> 01:30:52,996
If you ever introduce a third
master or something like that,

2066
01:30:53,526 --> 01:30:55,366
although MySQL though
wouldn't play nicely with that,

2067
01:30:55,536 --> 01:30:56,836
now you have to change
your code.

2068
01:30:56,836 --> 01:30:58,956
This is not a nice
layer of abstraction,

2069
01:30:59,376 --> 01:31:00,846
so how else could
we solve this, Axle?

2070
01:31:00,846 --> 01:31:05,106
I don't like the idea
of connecting each

2071
01:31:05,106 --> 01:31:07,686
of my web servers to the
database because frankly,

2072
01:31:07,916 --> 01:31:09,946
you know what, this is
going to get really ugly

2073
01:31:09,946 --> 01:31:12,156
if it starts looking
like this, right.

2074
01:31:12,216 --> 01:31:14,016
Very quickly this
degrades into a mess.

2075
01:31:14,746 --> 01:31:14,836
Yeah?

2076
01:31:15,366 --> 01:31:17,446
>> Get another machine,
load balancer [inaudible].

2077
01:31:17,706 --> 01:31:18,496
>> Okay, good.

2078
01:31:18,916 --> 01:31:20,586
>> [Inaudible] all of
that, so connect your --

2079
01:31:21,196 --> 01:31:24,786
your dub, dub, dub
servers to a load balancer,

2080
01:31:25,056 --> 01:31:26,846
then let it handle the request.

2081
01:31:26,846 --> 01:31:29,796
And then you can implement all
kinds of features like say,

2082
01:31:30,106 --> 01:31:33,246
they are both masters, right?

2083
01:31:33,246 --> 01:31:33,376
>> Uh-huh.

2084
01:31:33,376 --> 01:31:34,386
>> They have the same data.

2085
01:31:34,386 --> 01:31:34,766
>> Uh-huh.

2086
01:31:34,926 --> 01:31:38,566
>> What if only users with last
name that starts with M logs in,

2087
01:31:38,836 --> 01:31:41,796
well, you're going to have a
load on one particular server,

2088
01:31:41,976 --> 01:31:44,196
but then the load balancer
can take all those features we

2089
01:31:44,446 --> 01:31:45,446
talked about --

2090
01:31:45,676 --> 01:31:45,766
>> Good.

2091
01:31:45,766 --> 01:31:46,916
>> -- CPU cycles
and all of that,

2092
01:31:47,026 --> 01:31:50,796
and actually distribute MySQL
queries across databases.

2093
01:31:50,796 --> 01:31:51,846
>> Okay, good.

2094
01:31:51,846 --> 01:31:54,176
So we insert a load balancer
here, which is connected

2095
01:31:54,176 --> 01:31:55,486
to both the dub,
dub, dub machines

2096
01:31:55,486 --> 01:31:56,976
and also the database servers.

2097
01:31:57,256 --> 01:31:59,346
And then he can be
responsible for load balancing

2098
01:31:59,346 --> 01:32:00,596
across the two masters.

2099
01:32:00,596 --> 01:32:03,816
It's actually harder for
the database to do any kind

2100
01:32:03,816 --> 01:32:06,546
of intelligence load
balancing based on last names

2101
01:32:06,546 --> 01:32:09,556
at this point, since the MySQL
traffic is going to operate

2102
01:32:09,556 --> 01:32:13,496
with binary messages, not with
http style textual messages.

2103
01:32:13,886 --> 01:32:16,906
The load balancer up here
can look at HDP headers

2104
01:32:16,906 --> 01:32:18,176
and make intelligent decisions.

2105
01:32:18,456 --> 01:32:20,086
It's harder, maybe
not impossible,

2106
01:32:20,086 --> 01:32:22,776
but it wouldn't be very common
to do load balancing based

2107
01:32:22,776 --> 01:32:25,366
on application layer
intelligence here.

2108
01:32:25,366 --> 01:32:28,726
You would probably push that to
the PHP code again in that case.

2109
01:32:29,106 --> 01:32:31,296
But this isn't bad,
but Isaac doesn't

2110
01:32:31,296 --> 01:32:32,896
like this picture
now because of what?

2111
01:32:33,376 --> 01:32:34,206
>> It still fails [inaudible].

2112
01:32:35,396 --> 01:32:37,626
>> Yeah, so we still have
the single point of failure,

2113
01:32:37,626 --> 01:32:40,626
so you just cost me even more
money or more complexity,

2114
01:32:41,076 --> 01:32:42,536
even if I'm using free software.

2115
01:32:42,536 --> 01:32:43,796
This just takes more time now.

2116
01:32:44,166 --> 01:32:48,406
So now we have load balancer
one, load balancer two,

2117
01:32:48,776 --> 01:32:50,296
I need to do something
like this.

2118
01:32:51,136 --> 01:32:55,646
And even though this looks
a little ridiculous, well,

2119
01:32:55,646 --> 01:32:57,016
actually it's a little elegant.

2120
01:32:57,326 --> 01:32:57,946
That's pretty sexy.

2121
01:32:58,266 --> 01:33:01,396
So, you would do this
with switches or some kind

2122
01:33:01,396 --> 01:33:04,306
of Ethernet cables all going
into some central source.

2123
01:33:04,706 --> 01:33:06,816
So suppose instead
we actually did that?

2124
01:33:06,816 --> 01:33:08,986
If you've ever plugged in a
computer into a network jack,

2125
01:33:08,986 --> 01:33:11,466
which most of you probably
have even if you have a laptop,

2126
01:33:11,746 --> 01:33:14,046
you don't connect these
computers all to themselves.

2127
01:33:14,046 --> 01:33:16,856
You instead connect them to
like a big switch that has lots

2128
01:33:16,966 --> 01:33:18,796
of Ethernet ports that
you can plug into.

2129
01:33:19,426 --> 01:33:22,066
But now Isaac, what -- what do
you not like about this idea

2130
01:33:22,236 --> 01:33:23,726
if I'm plugging everything
into a switch?

2131
01:33:24,246 --> 01:33:26,686
>> It still fails.

2132
01:33:26,686 --> 01:33:29,776
>> Yeah, so welcome to the world
of like, network redundancy.

2133
01:33:30,036 --> 01:33:33,066
So really the right way to do
this is to have two switches,

2134
01:33:33,416 --> 01:33:36,466
so almost everyone of your
servers, database and web alike,

2135
01:33:36,466 --> 01:33:38,566
as well as you load
balancers, would typically have

2136
01:33:38,566 --> 01:33:40,886
at least two Ethernet
jacks in them.

2137
01:33:41,126 --> 01:33:42,776
And one cable would
go to one switch,

2138
01:33:42,776 --> 01:33:44,366
and another cable would
go to the other switch.

2139
01:33:44,366 --> 01:33:47,836
You have to be super careful not
to create loops of some sort.

2140
01:33:47,836 --> 01:33:50,196
So switches have to be
somewhat intelligent typically

2141
01:33:50,196 --> 01:33:52,196
so that you don't
create this crazy mess

2142
01:33:52,196 --> 01:33:53,826
where traffic is just
bouncing and bouncing

2143
01:33:53,826 --> 01:33:56,366
around in your internal network
and nothing's getting in or out.

2144
01:33:56,656 --> 01:33:59,586
So there's -- there's some
care that has to be taken,

2145
01:33:59,906 --> 01:34:02,926
but in general, this is
really the theme in ensuring

2146
01:34:02,926 --> 01:34:05,256
that you have not only
scalability, but redundancy

2147
01:34:05,256 --> 01:34:07,406
and higher probabilities
of uptime

2148
01:34:07,406 --> 01:34:08,666
and resilience against failure.

2149
01:34:09,046 --> 01:34:11,226
You really do start
cross connecting many

2150
01:34:11,226 --> 01:34:11,896
different things.

2151
01:34:12,276 --> 01:34:13,146
But let's push harder.

2152
01:34:13,146 --> 01:34:15,376
Isaac, what -- suppose
I fix the switch issue,

2153
01:34:15,636 --> 01:34:17,966
suppose I also make
this two load balancers

2154
01:34:17,966 --> 01:34:20,406
and fix that issue.

2155
01:34:20,606 --> 01:34:21,976
What's something else
that could fail now?

2156
01:34:32,076 --> 01:34:34,766
Can't do this on
an iPad very well.

2157
01:34:35,966 --> 01:34:37,506
So, this is your datacenter.

2158
01:34:38,056 --> 01:34:39,496
Here is the door
to your datacenter.

2159
01:34:40,956 --> 01:34:41,156
Jack?

2160
01:34:41,926 --> 01:34:43,266
>> The building burns down.

2161
01:34:43,556 --> 01:34:46,386
>> [Inaudible] good, more
extreme that I had in mind.

2162
01:34:46,386 --> 01:34:48,556
I was thinking the power
goes out, but that works too.

2163
01:34:48,556 --> 01:34:52,626
So the building itself
burns down or goes offline.

2164
01:34:52,626 --> 01:34:54,936
You have some kind of
network disconnect between you

2165
01:34:54,936 --> 01:34:59,236
and your ISP, the whole building
or the power indeed does go out.

2166
01:34:59,236 --> 01:35:00,046
And this has happened.

2167
01:35:00,046 --> 01:35:01,006
In fact, one of the things

2168
01:35:01,006 --> 01:35:03,506
that happens every
time Amazon goes out,

2169
01:35:03,506 --> 01:35:04,756
is the whole world
starts to think

2170
01:35:04,756 --> 01:35:08,536
that Cloud's cloud computing,
so to speak, is a -- the --

2171
01:35:08,536 --> 01:35:10,116
a bad thing because,
"Oh, my God.

2172
01:35:10,116 --> 01:35:11,346
Look, you can't keep
the cloud up."

2173
01:35:11,616 --> 01:35:13,546
But the tragedy here
is in this perception,

2174
01:35:13,546 --> 01:35:16,356
that cloud computing really
just refers to outsourcing

2175
01:35:16,356 --> 01:35:18,846
of services and sharing
resources like power

2176
01:35:18,846 --> 01:35:21,216
and networking and
security and so forth

2177
01:35:21,396 --> 01:35:23,046
across multiple customers.

2178
01:35:23,286 --> 01:35:26,106
So Amazon services EC2,
elastic compute cloud,

2179
01:35:26,376 --> 01:35:29,036
is kind of this picture here
whereby you don't own the

2180
01:35:29,036 --> 01:35:30,916
servers, but you do
rent space on them,

2181
01:35:30,916 --> 01:35:32,486
because they give
you VPSes that happen

2182
01:35:32,486 --> 01:35:33,956
to be housed inside
of this building.

2183
01:35:34,276 --> 01:35:36,506
Amazon offers things
called availability zones,

2184
01:35:36,506 --> 01:35:39,596
whereby this might be an
availability zone called US East

2185
01:35:39,596 --> 01:35:41,246
one, so this is a
building in Virginia,

2186
01:35:41,566 --> 01:35:42,576
in that particular case.

2187
01:35:42,856 --> 01:35:46,946
And what they offer though is
US East two and three and four,

2188
01:35:46,946 --> 01:35:49,686
though they call them A
and B and C and D. And what

2189
01:35:49,686 --> 01:35:52,626
that simply means in theory is
that there's another building

2190
01:35:52,626 --> 01:35:54,296
like this, drawn over there

2191
01:35:54,546 --> 01:35:56,606
that does not share
the same power source,

2192
01:35:56,606 --> 01:35:58,996
does not share the
same networking cables.

2193
01:35:58,996 --> 01:36:01,856
And so, even if something
goes wrong in one building,

2194
01:36:02,156 --> 01:36:04,106
in theory the other
shouldn't be affected.

2195
01:36:04,106 --> 01:36:05,826
However, Amazon has
suffered outages

2196
01:36:05,826 --> 01:36:08,986
in multiple availability
zones, multiple datacenters,

2197
01:36:09,346 --> 01:36:11,246
so in addition to having
servers in Virginia,

2198
01:36:11,246 --> 01:36:12,436
guess where else
they have servers?

2199
01:36:12,876 --> 01:36:16,996
>> Anywhere else.

2200
01:36:16,996 --> 01:36:17,836
>> Okay, anywhere else, Yeah,

2201
01:36:17,836 --> 01:36:18,946
that's actually a
pretty hard question,

2202
01:36:18,946 --> 01:36:19,626
the world's a big place.

2203
01:36:19,626 --> 01:36:22,676
So the West Coast and in
Asia, and in South America

2204
01:36:22,676 --> 01:36:23,986
and in Europe, as well.

2205
01:36:23,986 --> 01:36:26,636
They have different regions
as they call them inside

2206
01:36:26,636 --> 01:36:27,956
of which are different
datacenters

2207
01:36:27,956 --> 01:36:28,976
or availability zones.

2208
01:36:29,316 --> 01:36:32,336
But this just means that you
can really drive yourself crazy

2209
01:36:32,336 --> 01:36:34,326
thinking through all the
possible failure scenarios,

2210
01:36:34,326 --> 01:36:36,506
because even though
Jack's building burning

2211
01:36:36,506 --> 01:36:40,006
down is a little extreme, things
like that do happen, right.

2212
01:36:40,006 --> 01:36:43,046
If you have a massive storm,
like a tornado or a hurricane

2213
01:36:43,046 --> 01:36:44,356
that just knocks out power,

2214
01:36:44,566 --> 01:36:46,946
absolutely could a whole
building goes down -- go down.

2215
01:36:47,196 --> 01:36:48,486
So what do you do in that case?

2216
01:36:48,486 --> 01:36:51,736
Well, we have to have a second
datacenter or availability zone.

2217
01:36:51,736 --> 01:36:53,236
I'll draw it much
smaller this time,

2218
01:36:53,236 --> 01:36:54,966
even though it might be
this -- physically the same.

2219
01:36:55,406 --> 01:36:56,366
So here's another one.

2220
01:36:56,416 --> 01:36:58,526
Suppose that inside of
this building is exactly

2221
01:36:58,526 --> 01:36:59,586
that same topology,

2222
01:37:00,076 --> 01:37:04,026
so now really what we have is
the internet outside these boxes

2223
01:37:04,666 --> 01:37:06,006
connecting to both buildings.

2224
01:37:06,746 --> 01:37:09,876
So -- so internet is no
longer inside the building.

2225
01:37:10,966 --> 01:37:13,506
So, once you have
two datacenters,

2226
01:37:13,916 --> 01:37:17,326
how do you now distribute your
load across two datacenters?

2227
01:37:17,586 --> 01:37:17,936
Axle?

2228
01:37:18,266 --> 01:37:20,856
>> Well, then you can
use the DNS [inaudible].

2229
01:37:21,066 --> 01:37:23,576
>> Yeah, so we -- we didn't
really spend much time on it,

2230
01:37:23,576 --> 01:37:26,056
but recall that you can do load
balancing at the DNS level.

2231
01:37:26,056 --> 01:37:30,006
And this is indeed how you can
do geography based geolocate --

2232
01:37:30,306 --> 01:37:33,866
geography based load balancing,
whereby now when someone

2233
01:37:33,866 --> 01:37:37,646
on the internet requests the
IP address of something.com,

2234
01:37:37,966 --> 01:37:40,766
they might get the IP address
really of this building

2235
01:37:40,946 --> 01:37:43,576
or more specifically, of the
load balancer in this building.

2236
01:37:43,936 --> 01:37:46,116
Or they might get the IP
address of load balancer

2237
01:37:46,116 --> 01:37:47,786
in this building, right.

2238
01:37:47,786 --> 01:37:49,416
When we did the [inaudible]
lookup on Google,

2239
01:37:49,606 --> 01:37:50,786
we got a whole bunch of results.

2240
01:37:50,996 --> 01:37:53,166
That's not because they
have one building with lots

2241
01:37:53,166 --> 01:37:54,456
of load balancers inside of it.

2242
01:37:54,456 --> 01:37:57,436
That's because they probably
have lots of separate buildings

2243
01:37:57,486 --> 01:37:59,516
or datacenters, different
countries even,

2244
01:37:59,896 --> 01:38:02,046
that themselves have
different entry points,

2245
01:38:02,076 --> 01:38:03,166
different IP addresses.

2246
01:38:03,516 --> 01:38:05,636
So you have global
load balancing,

2247
01:38:05,636 --> 01:38:06,526
as it's typically called.

2248
01:38:06,706 --> 01:38:08,406
Then the request
comes into a building,

2249
01:38:08,406 --> 01:38:10,356
and you still have the
issue of somehow making sure

2250
01:38:10,356 --> 01:38:12,976
that subsequent traffic
gets to the same place,

2251
01:38:12,976 --> 01:38:14,936
because odds are Google is
not sharing your session

2252
01:38:14,936 --> 01:38:17,696
across entirely -- entirely
different continents.

2253
01:38:17,816 --> 01:38:18,986
Even though, they could be,

2254
01:38:19,226 --> 01:38:21,546
but that would probably be
expensive or slow to do.

2255
01:38:21,776 --> 01:38:24,066
So odds are you're going to stay
in that building for some amount

2256
01:38:24,066 --> 01:38:26,686
of time, but again, these
ideas we've been talking

2257
01:38:26,686 --> 01:38:28,396
about just get magnified
the bigger

2258
01:38:28,396 --> 01:38:29,926
and bigger you start to think.

2259
01:38:29,926 --> 01:38:31,976
And even then, you
have potential downtime

2260
01:38:32,196 --> 01:38:35,516
because if a whole building
goes offline and your browser

2261
01:38:35,516 --> 01:38:37,976
or your computer happens
to of cached the IP address

2262
01:38:38,536 --> 01:38:41,066
of that building,
that datacenter,

2263
01:38:41,366 --> 01:38:46,236
could take some minutes or some
hours for your TTL to expire

2264
01:38:46,466 --> 01:38:49,186
at which point you get
rerouted to something else.

2265
01:38:49,436 --> 01:38:50,736
Not too long ago,
just a few weeks,

2266
01:38:50,736 --> 01:38:52,006
I think Core [phonetic]
was offline

2267
01:38:52,006 --> 01:38:54,146
for several hours one
night because they Amazon.

2268
01:38:54,146 --> 01:38:56,636
A bunch of other
popular websites too,

2269
01:38:56,636 --> 01:38:59,376
who use Amazon services
were down altogether

2270
01:38:59,616 --> 01:39:02,826
because they were in a
building or a set of buildings

2271
01:39:02,826 --> 01:39:04,646
that suffered this
kind of downtime.

2272
01:39:04,646 --> 01:39:05,636
And it's hard.

2273
01:39:05,636 --> 01:39:08,116
Like if you are having
the fortunate problem

2274
01:39:08,116 --> 01:39:11,386
of way too many users and lots
of revenue, it gets harder

2275
01:39:11,386 --> 01:39:13,906
and harder to actually
scale things out globally.

2276
01:39:13,906 --> 01:39:15,426
So, typically people
do what they can,

2277
01:39:15,746 --> 01:39:17,816
but Isaac has gotten very
good at pointing out,

2278
01:39:17,816 --> 01:39:20,626
you can at least avoid as
best as possible these kinds

2279
01:39:20,626 --> 01:39:21,886
of single points of failure.

2280
01:39:24,456 --> 01:39:25,256
Questions?

2281
01:39:26,556 --> 01:39:27,756
So a word on security then.

2282
01:39:27,756 --> 01:39:28,936
Let's focus only
on this picture,

2283
01:39:28,936 --> 01:39:29,906
not so much on the building.

2284
01:39:30,286 --> 01:39:33,336
What kind of traffic now
needs to be allowed in

2285
01:39:33,336 --> 01:39:34,716
and out of the building?

2286
01:39:34,806 --> 01:39:37,906
So let me go ahead and just
give myself some internet here,

2287
01:39:38,506 --> 01:39:40,416
connecting to the load
balancers somehow.

2288
01:39:40,996 --> 01:39:44,176
What type of internet
traffic should be coming

2289
01:39:44,286 --> 01:39:47,106
from the outside world in
if I'm hosting a website,

2290
01:39:47,456 --> 01:39:49,176
a LAMP based website?

2291
01:39:49,246 --> 01:39:49,346
Yeah?

2292
01:39:49,946 --> 01:39:51,236
>> Well, you would
want a firewall

2293
01:39:51,236 --> 01:39:54,126
that allows only
port 80 connections.

2294
01:39:54,126 --> 01:39:54,696
>> Okay, good.

2295
01:39:54,696 --> 01:39:57,786
So I want TCP, recall
which is one

2296
01:39:57,786 --> 01:39:59,896
of the transport
protocols 80 on the way in.

2297
01:40:00,916 --> 01:40:04,456
That's good, but you just
compromised by ability

2298
01:40:04,456 --> 01:40:05,946
to have certain security, why?

2299
01:40:07,296 --> 01:40:09,086
You're not blocking a very
useful type of traffic.

2300
01:40:09,086 --> 01:40:10,226
>> Oh, yeah, the
encrypted [inaudible].

2301
01:40:10,536 --> 01:40:13,106
>> Good, so we also want 443 --

2302
01:40:13,266 --> 01:40:13,346
>> Yeah.

2303
01:40:13,346 --> 01:40:16,126
>> -- which is the default
port that's used for SSL,

2304
01:40:16,126 --> 01:40:18,216
for HTTPS based URL's.

2305
01:40:18,476 --> 01:40:19,016
So that's good.

2306
01:40:19,016 --> 01:40:20,936
This means now that the
only traffic allowed

2307
01:40:20,936 --> 01:40:23,946
into my datacenter
is TCCP80 and 443.

2308
01:40:23,946 --> 01:40:26,636
Now those familiar with SSH,
you've also just locked yourself

2309
01:40:26,636 --> 01:40:28,456
out of your datacenter,
because you cannot now SSH

2310
01:40:28,456 --> 01:40:31,046
into your datacenter, you
might want to allow something

2311
01:40:31,046 --> 01:40:34,406
like port 22, for
SSH or you might want

2312
01:40:34,556 --> 01:40:37,496
to have an SSL based VPN so
that you can connect somehow

2313
01:40:37,496 --> 01:40:38,846
to your datacenter remotely.

2314
01:40:38,966 --> 01:40:40,316
And again, it doesn't
have to be a datacenter,

2315
01:40:40,316 --> 01:40:42,036
this can just be some webhosting

2316
01:40:42,276 --> 01:40:44,366
or some VPS hosting
company that you're using.

2317
01:40:44,936 --> 01:40:48,266
And okay, so we might need one
or more other ports for our VPN,

2318
01:40:48,356 --> 01:40:49,506
but for now that's pretty good.

2319
01:40:49,826 --> 01:40:51,556
How about the load balancers?

2320
01:40:51,556 --> 01:40:52,976
What kind of traffic needs to go

2321
01:40:52,976 --> 01:40:55,716
from the load balancer
to my web servers?

2322
01:40:55,786 --> 01:41:04,086
Well, it -- it's really a
mess to keep it encrypted

2323
01:41:05,296 --> 01:41:07,496
because once it's inside the
datacenter, nobody else is going

2324
01:41:07,496 --> 01:41:10,186
to listen [inaudible] people
inside the datacenter.

2325
01:41:10,506 --> 01:41:10,656
>> Okay.

2326
01:41:10,656 --> 01:41:13,126
>> So you would want to drop
the encryption and do 80.

2327
01:41:13,576 --> 01:41:17,436
>> Good, and that's actually
very common to offload your SSL

2328
01:41:17,736 --> 01:41:20,156
to the load balancer
or some special device

2329
01:41:20,156 --> 01:41:22,376
and then keep everything
else unencrypted

2330
01:41:22,556 --> 01:41:24,686
because if you control
this, it's at least safer.

2331
01:41:24,686 --> 01:41:27,066
Not 100%, because if
someone compromises this now,

2332
01:41:27,066 --> 01:41:29,016
they're going to see
your traffic unencrypted.

2333
01:41:29,236 --> 01:41:32,666
But if you're okay with that,
doing the SSL termination here,

2334
01:41:32,826 --> 01:41:35,716
so everything's encrypted from
the internet down to here,

2335
01:41:35,826 --> 01:41:39,296
but then everything else goes
over normal unencrypted http.

2336
01:41:39,546 --> 01:41:41,816
The upside of that is remember
the whole certificate thing?

2337
01:41:42,126 --> 01:41:44,086
You don't need to put your
SSL certificate on all

2338
01:41:44,086 --> 01:41:47,016
of your web servers, you can
just put it in the load balancer

2339
01:41:47,016 --> 01:41:48,036
or the load balancers.

2340
01:41:48,336 --> 01:41:50,086
You can get expensive
load balancers

2341
01:41:50,086 --> 01:41:52,946
to handle the cryptography and
the computational cost thereof.

2342
01:41:52,946 --> 01:41:55,406
You can get cheaper web servers
because they don't need to worry

2343
01:41:55,406 --> 01:41:56,946
as much about that
kind of overhead.

2344
01:41:57,226 --> 01:41:58,056
So that's one option.

2345
01:41:58,056 --> 01:42:00,116
So TCP 80 here and here.

2346
01:42:00,386 --> 01:42:02,526
How about the traffic
between the web server

2347
01:42:02,526 --> 01:42:06,456
and the databases, perhaps
through these load balancers?

2348
01:42:07,006 --> 01:42:09,686
This is a more of
a trivia question.

2349
01:42:09,686 --> 01:42:12,856
But what kind of -- what
kind of traffic is it,

2350
01:42:12,856 --> 01:42:14,186
even if you don't
know the port number?

2351
01:42:15,156 --> 01:42:15,256
Yeah?

2352
01:42:15,936 --> 01:42:18,846
>> It's a -- it's
query or execute.

2353
01:42:18,936 --> 01:42:20,876
>> Yeah, query actually
-- or more specifically,

2354
01:42:20,876 --> 01:42:23,266
it's the SQL queries
like select and insert

2355
01:42:23,266 --> 01:42:24,836
and delete and so forth.

2356
01:42:25,126 --> 01:42:32,546
So this is generally TCP
3306, which is the port number

2357
01:42:32,546 --> 01:42:34,046
that MySQL uses by default.

2358
01:42:34,326 --> 01:42:35,296
So what does this mean?

2359
01:42:35,846 --> 01:42:38,306
If you do have firewalling
capabilities

2360
01:42:38,306 --> 01:42:40,916
and we haven't drawn any
firewalls per say, so we do need

2361
01:42:40,916 --> 01:42:44,376
to insert some hardware into
this picture that would allow us

2362
01:42:44,376 --> 01:42:46,766
to actually make these kinds
of configuration changes,

2363
01:42:46,796 --> 01:42:49,246
but if we assume we have
that ability in large part

2364
01:42:49,246 --> 01:42:51,316
because all of these things
are plugged in as we said

2365
01:42:51,316 --> 01:42:54,076
to some kind of switch, well
the switch could be a firewall

2366
01:42:54,076 --> 01:42:56,186
itself and we could make
these configuration changes.

2367
01:42:56,506 --> 01:42:59,566
We can further lock
things down, why?

2368
01:42:59,706 --> 01:43:02,546
I mean, everything just works
if I don't firewall things.

2369
01:43:03,786 --> 01:43:06,006
Why would I want to
bother tightening things

2370
01:43:06,006 --> 01:43:08,286
so that only 80 and
443 are allowed here,

2371
01:43:08,286 --> 01:43:10,246
and 3306 is allowed here?

2372
01:43:11,216 --> 01:43:14,046
And in fact, notice there's
no line between these guys.

2373
01:43:14,336 --> 01:43:18,656
>> Well, it would be really
stupid to keep, for example,

2374
01:43:18,656 --> 01:43:21,716
3306 open in the first
firewall because then people --

2375
01:43:22,246 --> 01:43:23,916
they might not be
able to because

2376
01:43:23,916 --> 01:43:26,206
of other security measures,
but in theory, they --

2377
01:43:26,206 --> 01:43:27,806
they are allowed by the firewall

2378
01:43:28,036 --> 01:43:32,336
to query your database
and do SQL commands.

2379
01:43:32,336 --> 01:43:32,736
>> Good, exactly.

2380
01:43:32,736 --> 01:43:34,916
There's just no need
for people to be able

2381
01:43:34,916 --> 01:43:37,846
to potentially even execute
SQL queries coming in

2382
01:43:37,846 --> 01:43:39,156
or make MySQL connections.

2383
01:43:39,276 --> 01:43:41,786
And even if you're not even
listening for MySQL connections,

2384
01:43:42,036 --> 01:43:43,746
it again is sort of the
principle of the thing.

2385
01:43:43,746 --> 01:43:46,276
You should really have the --

2386
01:43:46,276 --> 01:43:49,806
the principle of least privilege
whereby you only open those

2387
01:43:49,806 --> 01:43:52,286
doors that people actually
have to go through,

2388
01:43:52,286 --> 01:43:55,716
otherwise you're just
inviting unexpected behavior

2389
01:43:55,716 --> 01:43:57,536
because you left the
door ajar, so to speak.

2390
01:43:57,536 --> 01:44:00,996
You left the port open and it's
not clear whether someone might

2391
01:44:00,996 --> 01:44:02,226
in fact, take advantage of that.

2392
01:44:02,226 --> 01:44:05,586
Case and point, if somehow you
screw up or Apache screws up

2393
01:44:05,586 --> 01:44:08,456
or PHP screws up, and this
server is compromised,

2394
01:44:08,766 --> 01:44:12,796
it'd be kind of nice if the only
thing this server can do is talk

2395
01:44:12,796 --> 01:44:16,026
via MySQL to this server
and cannot for instance,

2396
01:44:16,026 --> 01:44:19,136
suddenly SSH to this
server or poke around

2397
01:44:19,136 --> 01:44:23,016
or execute any commands on
your network other than MySQL.

2398
01:44:23,016 --> 01:44:25,376
So at least if the bad guy
takes this machine over,

2399
01:44:25,596 --> 01:44:30,786
you really can't leave this
rectangle here that I've drawn.

2400
01:44:31,466 --> 01:44:34,256
So again, beyond the scope of
things we've done in the class,

2401
01:44:34,256 --> 01:44:36,586
and even though the appliance
itself actually does have a

2402
01:44:36,586 --> 01:44:38,846
firewall that allows
certain ports in and out,

2403
01:44:39,216 --> 01:44:41,696
all of the ones you need, we
haven't had to fine tune it

2404
01:44:41,996 --> 01:44:43,986
for any of the projects,
realize that you --

2405
01:44:44,256 --> 01:44:47,876
that even on something like a
LINUX based operating system.

2406
01:44:48,396 --> 01:44:50,146
So in short, as soon as
you have the happy problem

2407
01:44:50,146 --> 01:44:52,816
of having way too many
users for your own good,

2408
01:44:53,096 --> 01:44:55,916
lots of new problems arise
even though thus far be focused

2409
01:44:56,176 --> 01:44:58,266
pretty much entirely on the
software side of things.

2410
01:44:58,806 --> 01:45:01,126
So that is it for
computer science S75.

2411
01:45:01,126 --> 01:45:02,896
I'll stick around for
questions one on one.

2412
01:45:03,186 --> 01:45:06,116
We still have a final section
tonight, for those of you

2413
01:45:06,116 --> 01:45:08,596
who would like to dive
into some related topics,

2414
01:45:08,946 --> 01:45:13,506
otherwise realize that
the final project two,

2415
01:45:13,506 --> 01:45:14,496
it's deadline is coming up.

2416
01:45:14,496 --> 01:45:16,086
You should've gotten
feedback by --

2417
01:45:16,086 --> 01:45:17,476
from your TF's about
project one,

2418
01:45:17,476 --> 01:45:20,196
if not just drop him
or her a note or me.

2419
01:45:20,286 --> 01:45:21,806
Otherwise, it's been a pleasure
having everyone in the class.

2420
01:45:21,946 --> 01:45:24,036
We will see you online
after tonight [applause].

2421
01:45:24,296 --> 01:45:26,596
Oh, that's okay.

2422
01:45:29,476 --> 01:45:31,796
Thanks, I wasn't trying to
build up to that there, sorry.