1
00:00:09,810 --> 00:00:16,290
okay a viral i'm gonna talk about some
work often recently so for testing web

2
00:00:16,290 --> 00:00:20,670
services or all the testing the client
codes which we use and i still to

3
00:00:20,670 --> 00:00:21,600
web services

4
00:00:23,040 --> 00:00:29,480
so what is web service because of examples
to go contacts people you to i

5
00:00:29,480 --> 00:00:30,130
guess a web

6
00:00:30,670 --> 00:00:37,260
open street maps a few also downloading map files
facebook scruffy ple basically any hasty

7
00:00:37,260 --> 00:00:39,690
gpus H T S service which

8
00:00:40,880 --> 00:00:42,670
can be and is used by the desktop

9
00:00:43,320 --> 00:00:47,860
i don't specifically limiting my work
eight H T P and he cts because it

10
00:00:47,860 --> 00:00:48,930
makes everything a whole is

11
00:00:50,730 --> 00:00:51,910
and doesn't really

12
00:00:52,340 --> 00:00:53,940
restrictive anyway because

13
00:00:54,530 --> 00:00:58,390
pretty much every web service uses one of the same
because you don't get problems

14
00:00:58,390 --> 00:00:59,100
of a levels

15
00:00:59,710 --> 00:01:00,800
in that sense

16
00:01:02,230 --> 00:01:02,790
so

17
00:01:04,210 --> 00:01:07,450
when you think about testing web services
you go to the big question you can

18
00:01:07,450 --> 00:01:09,740
ones there is what you want to test

19
00:01:10,550 --> 00:01:14,010
you want to test your point code you
wanna make sure isn't crash but you

20
00:01:14,010 --> 00:01:19,700
want tested the point conforms to the current
behaviour of the server or some cost

21
00:01:19,700 --> 00:01:21,750
behaviour which is what you actually programmed against

22
00:01:22,070 --> 00:01:25,740
the user would say you want current behaviour
because that's what i actually want to

23
00:01:25,740 --> 00:01:31,200
use but that's quite tricky to do because
that means you gonna test online which

24
00:01:31,200 --> 00:01:38,260
gives you of those problems with connectivity and
race conditions and slowness and all sorts

25
00:01:39,370 --> 00:01:44,910
so we wanna catch regressions in point carried
but we also wanna catch that isn't

26
00:01:44,910 --> 00:01:49,670
it changes in the save as behaviour which
require changes in your point codes the

27
00:01:49,670 --> 00:01:50,690
new features or

28
00:01:51,260 --> 00:01:52,630
whatever the changed online

29
00:01:56,860 --> 00:01:57,450
so

30
00:01:58,350 --> 00:02:02,320
you can do it two ways you can do offline
testing or online testing and

31
00:02:02,320 --> 00:02:05,820
one talking about here is offline
testing developing working on

32
00:02:08,460 --> 00:02:12,290
the bait basic idea of offline testing is
that you somehow record how the server

33
00:02:12,290 --> 00:02:18,400
hey and then test against are offline not
using that work not touching the curve

34
00:02:18,400 --> 00:02:24,540
javier the server so it's reliable because younger
always network problem you've eliminated the web

35
00:02:24,540 --> 00:02:28,880
service and all the network infrastructure
in between from your system on the test

36
00:02:29,380 --> 00:02:31,130
so massively cut down a good place

37
00:02:32,050 --> 00:02:36,360
but it's not what users care about because
the web service changes your point code

38
00:02:36,360 --> 00:02:38,040
isn't getting tested against like situation

39
00:02:40,550 --> 00:02:42,260
you can also

40
00:02:42,520 --> 00:02:44,640
test against the carburetor the server

41
00:02:45,110 --> 00:02:47,860
which is as i said what we use just
care about but it requires a

42
00:02:47,860 --> 00:02:53,610
network connections the instantly eliminates any kind
of continuous integration or testing on a seven

43
00:02:53,610 --> 00:02:59,200
provided by know because it's just so unreliable
it's hassles that so and network admins

44
00:02:59,200 --> 00:02:59,810
don't like it

45
00:03:00,660 --> 00:03:01,720
it's unreliable

46
00:03:02,730 --> 00:03:06,770
and that's a problem that's it's
and me a lot in the G data which

47
00:03:06,770 --> 00:03:11,210
is more based this work on where basically
every time a run make check it

48
00:03:11,210 --> 00:03:13,260
failed and that would be D to

49
00:03:13,920 --> 00:03:19,750
transient network problems or subtle changes
in the so that it behaviour that S how

50
00:03:19,750 --> 00:03:22,760
to compensated for but the point was actually point

51
00:03:25,300 --> 00:03:30,310
another thing about online testing is that
web services change so if you're gonna do

52
00:03:30,310 --> 00:03:32,130
offline testing you need to

53
00:03:32,580 --> 00:03:37,140
be able to change your tests and update
them so they represent the lexus takes

54
00:03:37,140 --> 00:03:40,540
of a server and that was easy because
you don't wanna spend time doing that

55
00:03:40,540 --> 00:03:41,780
you got much better things today

56
00:03:44,860 --> 00:03:49,630
finally what services are distributed systems
which is a problem it's an lots

57
00:03:50,140 --> 00:03:55,290
which means that roger circuit a good wasn't
example later just have one server which

58
00:03:55,290 --> 00:03:59,870
has a leo contacts on they have hundreds
and as it turns out if you

59
00:03:59,870 --> 00:04:04,120
say for example instead to contact in your
address book and it says another one

60
00:04:04,120 --> 00:04:07,000
and then query your address book one you contacts

61
00:04:07,610 --> 00:04:11,770
often they don't here for ten twenty seconds

62
00:04:12,180 --> 00:04:15,800
which means that your test code has to get
to sleep the channel twenty seconds

63
00:04:15,800 --> 00:04:20,570
and that's it means your tests take ages
to run which is horrible because you

64
00:04:20,570 --> 00:04:24,460
can get most of the way three make
check and then it'll five and you

65
00:04:24,460 --> 00:04:25,370
gotta start again

66
00:04:26,790 --> 00:04:29,960
so that's one of the major reasons why online testing bad

67
00:04:33,980 --> 00:04:35,670
there are also other problems with

68
00:04:37,350 --> 00:04:41,790
online testing do you to things like resource limits and

69
00:04:42,250 --> 00:04:46,110
the fact that you have to use not a natural
online account so testing against

70
00:04:46,110 --> 00:04:50,790
cable services they limit you to say a thousand
press today so if you wanna

71
00:04:50,790 --> 00:04:55,200
run make checking continuous integration you gonna
hit against that limits and also new tests

72
00:04:55,200 --> 00:04:57,920
are gonna fail for us today and there's
nothing you can do about that

73
00:04:58,690 --> 00:05:03,600
similarly if you wanna run test from multiple
locations so one developer in europe and

74
00:05:03,600 --> 00:05:09,310
one america and they both whatever make czech
good people is gonna say well that

75
00:05:09,310 --> 00:05:11,660
test account is be involved in two places at once

76
00:05:12,330 --> 00:05:15,570
i'm gonna look at out and then
again make czech stops working

77
00:05:17,580 --> 00:05:20,070
so it would be working on is

78
00:05:20,960 --> 00:05:24,590
maybe oscillation for offline testing

79
00:05:25,290 --> 00:05:29,890
where you record H T P messages between
client and server you save into a

80
00:05:29,890 --> 00:05:34,260
file you commit that get you then run only
a test again start but periodically

81
00:05:34,260 --> 00:05:40,500
you can manually check and compare the trace
files against the carburetor was of the

82
00:05:40,500 --> 00:05:44,310
and check other the same or if the difference
then you know what's changed so

83
00:05:44,310 --> 00:05:48,740
you cannot take your code in your tests
to match but it means that's for

84
00:05:48,740 --> 00:05:52,710
most the time this of one testing can be
done in continuous integration it can

85
00:05:52,710 --> 00:06:00,200
be a lot faster sleep and you don't have always
problems with synchronization or resource

86
00:06:00,200 --> 00:06:00,670
limits

87
00:06:04,150 --> 00:06:09,860
so there are various other existing solutions
i mean it's a pretty simple idea but

88
00:06:09,860 --> 00:06:14,530
most of the large over only so the first three
they basically exactly what describes

89
00:06:16,170 --> 00:06:21,020
why more is a separate process which
makes in the paying to set up

90
00:06:21,470 --> 00:06:25,570
rest drive a is rest only which eliminates
a huge number one services but

91
00:06:25,980 --> 00:06:30,200
also means you can test if you don't know
number one services and betamax children

92
00:06:30,200 --> 00:06:30,720
as well

93
00:06:31,440 --> 00:06:36,090
question they will smoke an e-mail of all
the open source projects which is similar

94
00:06:36,090 --> 00:06:40,790
sort of things with the cost and hardware but
obviously not exceeding gonna translate and

95
00:06:40,790 --> 00:06:41,540
fractions

96
00:06:43,900 --> 00:06:48,360
so the approach of taking would look
at it is to a broad enough is

97
00:06:48,360 --> 00:06:49,400
a revelation

98
00:06:50,640 --> 00:06:56,760
we directly apply requests to a local in precisely
but other and then from that

99
00:06:56,760 --> 00:07:01,860
replay the trace and compare every incoming
H T you request with the trains and

100
00:07:01,860 --> 00:07:03,890
then reply with respondent race well

101
00:07:04,500 --> 00:07:07,580
if they don't if they you can make the press
that match what's in that

102
00:07:07,580 --> 00:07:12,740
race while then you get never your tests
fail and everyone stopping one happy

103
00:07:14,140 --> 00:07:18,520
alternatively you can not a variety S
and the test will go straight to the

104
00:07:18,520 --> 00:07:23,900
online server and then you can compare the responses
not with the choice while and

105
00:07:23,900 --> 00:07:28,200
see if the servers changed compared
to last time you save the choices

106
00:07:29,480 --> 00:07:34,160
so you this operates at H T P level
using them so you which is

107
00:07:34,160 --> 00:07:37,280
simple things a lot this transpose
can be human readable and

108
00:07:37,920 --> 00:07:39,260
does debug able

109
00:07:40,970 --> 00:07:42,530
it means you can do

110
00:07:44,010 --> 00:07:49,750
simulation of H T P errors like timeouts
and for a forcible server iras every

111
00:07:49,750 --> 00:07:54,160
five hundreds things like that with relative
ease you not filling around of the lower

112
00:07:54,160 --> 00:07:56,790
levels of T C P I P which would just be too much for fire

113
00:07:56,790 --> 00:08:00,480
it gives you extra power but for the most
part it's not needed for more

114
00:08:00,480 --> 00:08:02,630
web services "'cause" there will be H T to you know like

115
00:08:04,830 --> 00:08:06,720
unlike a one or more

116
00:08:07,340 --> 00:08:13,200
the server that's your testing against the offline
server is kept in process so you

117
00:08:13,200 --> 00:08:19,610
don't have process synchronisation signal tests
and this mock server so some for things a

118
00:08:19,610 --> 00:08:20,130
lot

119
00:08:23,430 --> 00:08:28,340
and because it's overriding the N S it requires
minimal code changes to your point

120
00:08:28,340 --> 00:08:31,580
so of the solutions that people

121
00:08:32,380 --> 00:08:36,510
suggest would be to take your clients and
split it out into interface isn't and

122
00:08:36,510 --> 00:08:42,390
reimplement interfaces with an online petition offline
version and that becomes fourty complicated cost many

123
00:08:42,390 --> 00:08:43,580
changes and is

124
00:08:44,420 --> 00:08:45,260
a real fast

125
00:08:47,090 --> 00:08:51,660
so by writing the N S you basically
have to change a client to

126
00:08:52,860 --> 00:08:56,400
specimen H T people to use because
it'll be a wide range one so you'd

127
00:08:56,400 --> 00:09:01,240
have to run three but you can easily switch
between online and offline just by

128
00:09:01,240 --> 00:09:02,710
turning off the gas reservation

129
00:09:04,990 --> 00:09:07,260
so diagrams everyone much diagrams

130
00:09:07,880 --> 00:09:11,440
so this is have things currently all you've
got your system on the test which

131
00:09:11,440 --> 00:09:14,920
is your point code running on the desktop
and you've got on one server which

132
00:09:14,920 --> 00:09:18,820
are testing against you got always this page
T V transactions going between them

133
00:09:20,330 --> 00:09:21,020
so

134
00:09:21,420 --> 00:09:26,570
now introduce a lots of the choice
while and you locally say C T V

135
00:09:26,570 --> 00:09:31,730
transactions to that race while so we still
doing online stuff but once when you

136
00:09:31,730 --> 00:09:34,770
first set of tests you save more to try as well

137
00:09:35,450 --> 00:09:36,810
and commitment to get

138
00:09:38,690 --> 00:09:42,870
you can and the your testing case that race
while eliminate on one server everything

139
00:09:42,870 --> 00:09:48,720
goes remote server by redirecting guess the choice
well get right into that and of

140
00:09:48,720 --> 00:09:49,210
course happy

141
00:09:50,680 --> 00:09:55,160
you can also compare your choice well against
the online so the point again switching

142
00:09:55,160 --> 00:10:00,430
to doing the transactions online but also reading
interest well i'm comparing now same time

143
00:10:01,320 --> 00:10:05,790
so you can see what's changed you do not
manually once a month or whatever

144
00:10:05,790 --> 00:10:09,560
one of if you like it or something develop
to take trough and you continuous

145
00:10:09,560 --> 00:10:10,210
integration so

146
00:10:12,420 --> 00:10:15,060
so he's a example

147
00:10:15,690 --> 00:10:19,680
of how we change your unit test so the test it is here

148
00:10:20,350 --> 00:10:26,370
whatever it is something sensation he request gets
response pulses it doesn't ounce bottle of

149
00:10:26,370 --> 00:10:28,250
and what you have is

150
00:10:29,010 --> 00:10:32,760
beginning dedicated and dedicated around each unit test

151
00:10:33,670 --> 00:10:36,460
which was a trace the stock price

152
00:10:36,860 --> 00:10:42,380
it's cool since your code somewhat susceptible
to use because you gonna use this high

153
00:10:42,380 --> 00:10:45,340
range pull so you don't wanna use
rate don't just use port eighty

154
00:10:47,800 --> 00:10:53,360
and then you also tell the D N S
resolver which your is to have

155
00:10:53,360 --> 00:10:57,570
a right so if you testing able services
you know growing people dot com and

156
00:10:57,570 --> 00:11:01,630
some supplements that when you set up
to run results you'll be back seven

157
00:11:02,350 --> 00:11:08,610
this also means that any requests that you don't
expect to other domains however unlikely

158
00:11:08,610 --> 00:11:12,940
that they would gets a D N S translation
arrow and so you can catch

159
00:11:12,940 --> 00:11:14,030
buttons that way

160
00:11:15,140 --> 00:11:16,000
probably not gonna happen

161
00:11:17,420 --> 00:11:21,140
and you can wrap as a lot than us
and macros or something so you

162
00:11:21,140 --> 00:11:23,970
basically just a one line before and after each unit test

163
00:11:25,110 --> 00:11:30,730
there is some code i'm not showing here
which you and you log handler which

164
00:11:30,730 --> 00:11:36,440
will be htk mess in haiti messages like
tricks while but that's just a few

165
00:11:36,440 --> 00:11:37,850
lines didn't feel

166
00:11:40,960 --> 00:11:45,240
so i've implemented this village a data
which is the project a hard on about

167
00:11:45,240 --> 00:11:46,050
most

168
00:11:47,310 --> 00:11:51,300
so make czech was over ten minutes because
of all these sleeps to wait for

169
00:11:51,300 --> 00:11:54,810
the synchronisation in the street
a system on the server side

170
00:11:55,360 --> 00:11:59,480
you can emanate willows and make czech negatives
down two hundred and thirty seven seconds

171
00:12:00,420 --> 00:12:02,380
done with a quick run of point make check

172
00:12:03,210 --> 00:12:08,660
you can cover interesting about the same after implementing
this it's likely increased because i

173
00:12:08,660 --> 00:12:12,980
did a few tests for every conditions
which i can nowadays so i can now

174
00:12:12,980 --> 00:12:17,840
simulates and this of the giving you an access
denied error or saying your accounts

175
00:12:17,840 --> 00:12:19,240
been compromised

176
00:12:20,130 --> 00:12:22,460
so you can add test cases for those
which you wouldn't be able to do

177
00:12:22,460 --> 00:12:23,370
for

178
00:12:24,560 --> 00:12:26,110
because you can't manipulate so state

179
00:12:27,690 --> 00:12:29,540
but generally it's been useful

180
00:12:31,440 --> 00:12:32,090
i haven't

181
00:12:32,750 --> 00:12:36,480
applied the store the libraries yes but there's
a whole later than can be modified

182
00:12:36,480 --> 00:12:42,140
this that and help also there's the facebook
graphic the only as a go a

183
00:12:42,140 --> 00:12:44,760
do you able a no one counts

184
00:12:46,350 --> 00:12:49,570
do the network as it could possibly use it as well this

185
00:12:49,880 --> 00:12:52,610
you know as a lot of stuff that could have it and it was is

186
00:12:52,610 --> 00:12:53,570
become a bit more stable

187
00:12:55,530 --> 00:12:58,230
but there are some problems so

188
00:12:59,720 --> 00:13:03,680
in your traditional unit tests that's
a you have a set of function the unit

189
00:13:03,680 --> 00:13:07,820
test function attend on function and you can
reuse the set of content and functions

190
00:13:07,820 --> 00:13:12,650
for multiple unit tests because they do
common things like you have won the ads

191
00:13:12,650 --> 00:13:16,680
the contact and that at a functional remove
contacts and then you have there is

192
00:13:16,680 --> 00:13:20,420
unit tests do different things with a new contract

193
00:13:21,910 --> 00:13:23,370
unfortunately this means that's

194
00:13:23,770 --> 00:13:25,090
if you have a choice while

195
00:13:26,530 --> 00:13:27,690
per unit test

196
00:13:28,390 --> 00:13:29,940
and they have point these in them

197
00:13:30,800 --> 00:13:33,150
not identify your contact for example

198
00:13:34,810 --> 00:13:38,490
the only these are generated by the server
will differ from indication of set up

199
00:13:38,490 --> 00:13:40,080
to different invocations that so

200
00:13:40,850 --> 00:13:44,600
and there's a bit of written in the trace
files which means that your unit

201
00:13:44,600 --> 00:13:45,480
tests will need

202
00:13:46,370 --> 00:13:51,210
well that response for each unit test believe
modifying afterwards to make a list id

203
00:13:51,210 --> 00:13:51,750
is the same

204
00:13:52,480 --> 00:13:55,730
which is something i haven't worked out how to fix yet

205
00:13:56,920 --> 00:13:58,200
probably isn't too hard

206
00:13:59,010 --> 00:14:03,660
similarly you go from the time stamps
if you have H T P response which

207
00:14:03,660 --> 00:14:07,990
includes time stamps well in the body
or in that is a nice it up

208
00:14:07,990 --> 00:14:11,530
to choice while and then you run test
against it next month it's on some

209
00:14:11,530 --> 00:14:14,970
so gonna get acetate and i could
cause problems in some cases

210
00:14:16,080 --> 00:14:17,210
again i think is work around

211
00:14:19,580 --> 00:14:23,500
and then also if you may detest against cancellation
the messages the current way of

212
00:14:23,500 --> 00:14:25,320
will be noticed that race problems knots

213
00:14:26,020 --> 00:14:27,380
particularly great

214
00:14:29,680 --> 00:14:30,890
and needs a bit of work

215
00:14:33,140 --> 00:14:37,940
so currently this work is being done
in the G data and i haven't yet

216
00:14:37,940 --> 00:14:41,360
split up into separate project i'm in the process
of doing that and it'll be

217
00:14:41,360 --> 00:14:43,020
committed to get doris shortly

218
00:14:44,980 --> 00:14:46,880
sorry please on other people can start using it

219
00:14:49,100 --> 00:14:54,950
there's a lot of potential running standard error
responses way like return an error five

220
00:14:54,950 --> 00:15:00,090
hundred or narrow for a for or whatever
and it be good to implements

221
00:15:00,390 --> 00:15:02,020
some kind of easy support for that in they

222
00:15:02,880 --> 00:15:06,960
the testing code so that every point code
uses it doesn't have to read that

223
00:15:07,620 --> 00:15:10,240
and goals need to what of the libraries

224
00:15:11,780 --> 00:15:13,600
so that was

225
00:15:14,390 --> 00:15:18,450
representation of it and also construct a bit
of a discussion about this because it's

226
00:15:18,450 --> 00:15:22,960
an area when i'm can improve a lot basically
most of a networking code websites

227
00:15:22,960 --> 00:15:23,610
is good is

228
00:15:24,190 --> 00:15:26,580
effectively untested that lines so

229
00:15:27,260 --> 00:15:32,020
this is a like a brief implementation
of my ideas and i'm sure there's a

230
00:15:32,020 --> 00:15:34,210
lot of stuff can be added an improved

231
00:15:34,700 --> 00:15:37,100
and i'd welcome any suggestions people have

232
00:15:37,670 --> 00:15:38,470
well questions

233
00:15:51,420 --> 00:15:56,670
i do you deal with the be catching
a bit encrypted connections

234
00:15:58,140 --> 00:16:00,500
as in H T us encryption

235
00:16:01,750 --> 00:16:03,520
catching

236
00:16:04,450 --> 00:16:09,060
well if i understood well the moat server cash the request

237
00:16:09,640 --> 00:16:14,810
yes that catches the H T P requests so actually yes

238
00:16:16,380 --> 00:16:22,430
packaging of that this is not don't within
the tries postal remote server has its

239
00:16:22,430 --> 00:16:28,670
own dummy certificates and K which you
have to commit to get and then you

240
00:16:28,670 --> 00:16:33,240
also have to model for your point code
to say no don't districts of course

241
00:16:33,240 --> 00:16:39,180
in checks we so there are being work
that for that kind of like so

242
00:16:39,180 --> 00:16:43,020
you've done some work in yes i mean
would you know to use it H

243
00:16:43,020 --> 00:16:49,530
T yes and also working point obviously you modifications
district see a checks would be

244
00:16:49,530 --> 00:16:52,290
the okay like and should never be used in production but

245
00:16:52,820 --> 00:16:54,130
this is a few lines

246
00:17:07,460 --> 00:17:12,660
so i wanted to ask you to will who
are offering some kind of staging

247
00:17:12,660 --> 00:17:17,240
services like to just it on production servers or

248
00:17:18,690 --> 00:17:20,140
is there something else

249
00:17:21,480 --> 00:17:26,020
so i don't know about anything else but giggle
don't have testing service they have

250
00:17:26,020 --> 00:17:31,490
an online api playground which is web
only when you can talk in requests on

251
00:17:31,490 --> 00:17:37,270
a web page and test them out but that's
mostly documentation thing so what the

252
00:17:37,270 --> 00:17:40,830
test cases that i have run against productions
others and that's what you run into

253
00:17:40,830 --> 00:17:43,680
these problems with and request limits and

254
00:17:43,980 --> 00:17:45,200
accounting not so

255
00:17:45,820 --> 00:17:47,900
and it would be nice if they provided

256
00:17:48,500 --> 00:17:50,890
playground sins a staging service

257
00:17:51,840 --> 00:17:52,390
like that

258
00:18:01,620 --> 00:18:08,780
no questions okay so because will be upon
victorious hopefully in the next day at

259
00:18:08,780 --> 00:18:13,220
the moment it's in the G data in the tests
directory and wants to take

260
00:18:13,220 --> 00:18:16,600
a look and or controlled we have to is about me

261
00:18:17,010 --> 00:18:19,200
thoughts you have so thank you much

262
00:18:19,910 --> 00:18:20,200
thank you