Coarse coding和function approximation

Coarse coding
coarse coding可以影响泛化和分辨的性能。
The size, number and shape of features affect generalization.
The resulting shape interactions affect the ability to discrimination.

Exploration in function approximation
Optimistic initial value should be localized in function approximation. 想象如果只有一个特征,那么无论怎样都会被更新，也就是说一开始的初始值无论怎么样都会减少，也就是失去了optimistic value的效果,所以更新应当是局部的才有效。但是往往neural network will lose optimistic values quickly。

average reward
Many discounted reward can be written in average reward setting.