this repo has no description
Continue rapport
+69
-74
+69
-74
rapport/proofs.typ
+69
-74
rapport/proofs.typ
···
80
80
On a
81
81
82
82
$
83
-
forall t in NN, quad bb(P)(A_t = a_t mid(|) S_t = s_t) = Q_p (s_t, a_t)
84
-
$
85
-
86
-
Or
87
-
88
-
$
83
+
bb(P)(S_0 = s_0) &= rho_0(s_0) \
84
+
forall t in NN, quad bb(P)(A_t = a_t mid(|) S_t = s_t) &= Q_p (s_t, a_t) \
89
85
forall t in NN^*, quad
90
86
bb(P)(S_t = s_t | C_(t-1) = c_(t-1)) &= bb(P)(M(C_(t-1)) = M(c_(t-1)) | C_(t-1) = c_(t-1)) \
91
-
&= bb(P)(C_(t-1) = c_(t-1) | C_(t-1) = c_(t-1)) \
92
-
&= 1
87
+
&= bb(P)(C_(t-1) = c_(t-1) | C_(t-1) = c_(t-1)) = 1
93
88
$
94
89
95
-
Et
96
-
97
-
$
98
-
bb(P)(S_0 = s_0) = rho_0(s_0)
99
-
$
100
90
101
91
Donc on a
102
92
···
119
109
120
110
#let clip = $op("clip")$
121
111
122
-
Soit $(s, a) in S times A$, et $cal(P)'$ une politique.
123
-
124
-
Pour alléger les notations, posons $
125
-
alpha &:= A_(cal(P)', R) (s, a) \
126
-
q &:= Q_cal(P) (s, a) \
127
-
q' &:= Q_cal(P)' (s, a) \
128
-
$
112
+
Soit $(s, a) in S times A$, et $cal(P)'$ une politique. Posons $alpha &:= A_(cal(P)', R) (s, a)$, $q slash q' &:= Q_cal(P) (s, a) slash Q_cal(P)' (s, a)$ .
129
113
130
-
#let why = explanation => $\ quad & #[car #explanation]$
114
+
#let why = explanation => $\ & quad quad #[car #explanation]$
131
115
132
116
#{
117
+
v(1.5em)
133
118
set math.equation(numbering: none)
119
+
show math.equation.where(block: true): set align(left)
120
+
block(breakable: false,
134
121
grid(columns: (1fr, 1fr), row-gutter: 1em,
135
-
[
136
122
137
-
*Cas $alpha > 0$ et $q slash q' in [1-epsilon, 1+epsilon]$*
123
+
grid.cell(align: center)[ *Cas $alpha > 0$* ],
124
+
grid.cell(align: center)[ *Cas $alpha < 0$* ],
125
+
[
138
126
139
-
$
140
-
&L(s, a, cal(P), cal(P'), R) \
141
-
&= min(q/q' alpha, quad clip(q/q', thick 1-epsilon, thick 1+epsilon) alpha) \
142
-
&= min(q/q', quad clip(q/q', thick 1-epsilon, thick 1+epsilon) ) alpha why(alpha > 0) \
143
-
&= min(q/q', quad q/q') alpha \
144
-
&= min(q/q' , 1+epsilon) alpha \
145
-
$
146
-
], [
127
+
$
128
+
&L(s, a, cal(P), cal(P'), R) \
129
+
&= min(q/q' alpha, quad clip(q/q', thick 1-epsilon, thick 1+epsilon) alpha) \
130
+
&= min(q/q', quad clip(q/q', thick 1-epsilon, thick 1+epsilon) ) alpha why(alpha > 0) \
131
+
$
132
+
],
133
+
[
147
134
148
-
*Cas $alpha > 0$ et $q slash q' > 1+epsilon$*
135
+
$
136
+
&L(s, a, cal(P), cal(P'), R) \
137
+
&= min(q/q' alpha, quad clip(q/q', thick 1-epsilon, thick 1+epsilon) alpha) \
138
+
&= max(q/q', quad clip(q/q', thick 1-epsilon, thick 1+epsilon) ) alpha why(alpha < 0) \
139
+
$
140
+
],
149
141
150
-
$
151
-
&L(s, a, cal(P), cal(P'), R) \
152
-
&= min(q/q' alpha, quad clip(q/q', thick 1-epsilon, thick 1+epsilon) alpha) \
153
-
&= min(q/q', quad clip(q/q', thick 1-epsilon, thick 1+epsilon) ) alpha why(alpha > 0) \
154
-
&= min(q/q', quad 1+epsilon) alpha \
155
-
$
142
+
grid.hline(stroke: 0.5pt),
143
+
grid.cell(colspan: 2, align: center, inset: 1em)[*...et $q slash q' in [1-epsilon, 1+epsilon]$*],
144
+
[
145
+
146
+
$
147
+
&= min(q/q', quad clip(q/q', thick 1-epsilon, thick 1+epsilon) ) alpha \
148
+
&= min(q/q', quad q/q') alpha \
149
+
&= min(q/q' , 1+epsilon) alpha \
150
+
$
151
+
156
152
157
-
], [
153
+
], [
158
154
159
-
*Cas $alpha > 0$ et $q slash q' < 1-epsilon$*
155
+
$
156
+
&= max(q/q', quad clip(q/q', thick 1-epsilon, thick 1+epsilon) ) alpha \
157
+
&= max(q/q', quad q/q') alpha \
158
+
&= max(q/q' , 1-epsilon) alpha \
159
+
$
160
160
161
-
$
162
-
&L(s, a, cal(P), cal(P'), R) \
163
-
&= min(q/q' alpha, quad clip(q/q', thick 1-epsilon, thick 1+epsilon) alpha) \
164
-
&= min(q/q', quad clip(q/q', thick 1-epsilon, thick 1+epsilon) ) alpha why(alpha > 0) \
165
-
&= min(q/q', quad 1-epsilon) alpha \
166
-
&= min(q/q', quad 1+epsilon) alpha why(1+epsilon > 1-epsilon > q / q') \
167
-
$
161
+
],
168
162
169
-
], [
163
+
grid.hline(stroke: 0.5pt),
164
+
grid.cell(colspan: 2, align: center, inset: 1em)[*...et $q slash q' > 1+epsilon$*],
165
+
[
170
166
171
-
*Cas $alpha < 0$ et $q slash q' in [1-epsilon, 1+epsilon]$*
167
+
$
168
+
&= min(q/q', quad clip(q/q', thick 1-epsilon, thick 1+epsilon) ) alpha \
169
+
&= min(q/q', quad 1+epsilon) alpha \
170
+
$
172
171
173
-
$
174
-
&L(s, a, cal(P), cal(P'), R) \
175
-
&= min(q/q' alpha, quad clip(q/q', thick 1-epsilon, thick 1+epsilon) alpha) \
176
-
&= max(q/q', quad clip(q/q', thick 1-epsilon, thick 1+epsilon) ) alpha why(alpha < 0) \
177
-
&= max(q/q', quad q/q') alpha \
178
-
&= max(q/q' , 1-epsilon) alpha \
179
-
$
172
+
], [
180
173
181
-
], [
174
+
$
175
+
&= max(q/q', quad clip(q/q', thick 1-epsilon, thick 1+epsilon) ) alpha \
176
+
&= max(q/q', quad 1+epsilon) alpha \
177
+
&= max(q/q', quad 1-epsilon) alpha why(1-epsilon < 1+epsilon < q / q') \
178
+
$
182
179
183
-
*Cas $alpha < 0$ et $q slash q' > 1+epsilon$*
180
+
],
184
181
185
-
$
186
-
&L(s, a, cal(P), cal(P'), R) \
187
-
&= min(q/q' alpha, quad clip(q/q', thick 1-epsilon, thick 1+epsilon) alpha) \
188
-
&= max(q/q', quad clip(q/q', thick 1-epsilon, thick 1+epsilon) ) alpha why(alpha < 0) \
189
-
&= max(q/q', quad 1+epsilon) alpha \
190
-
&= max(q/q', quad 1-epsilon) alpha why(1-epsilon < 1+epsilon < q / q') \
191
-
$
182
+
grid.hline(stroke: 0.5pt),
183
+
grid.cell(colspan: 2, align: center, inset: 1em)[*...et $q slash q' < 1-epsilon$*],
184
+
[
192
185
193
-
], [
186
+
$
187
+
&= min(q/q', quad clip(q/q', thick 1-epsilon, thick 1+epsilon) ) alpha \
188
+
&= min(q/q', quad 1-epsilon) alpha \
189
+
&= min(q/q', quad 1+epsilon) alpha why(1+epsilon > 1-epsilon > q / q') \
190
+
$
194
191
195
-
*Cas $alpha < 0$ et $q slash q' < 1-epsilon$*
192
+
], [
196
193
197
-
$
198
-
&L(s, a, cal(P), cal(P'), R) \
199
-
&= min(q/q' alpha, quad clip(q/q', thick 1-epsilon, thick 1+epsilon) alpha) \
200
-
&= max(q/q', quad clip(q/q', thick 1-epsilon, thick 1+epsilon) ) alpha why(alpha < 0) \
201
-
&= max(q/q', quad 1-epsilon) alpha
202
-
$
203
194
204
-
])
195
+
$
196
+
&= max(q/q', quad clip(q/q', thick 1-epsilon, thick 1+epsilon) ) alpha \
197
+
&= max(q/q', quad 1-epsilon) alpha
198
+
$
205
199
200
+
]))
206
201
}