Concrete|Concrete dropout
一种采用贝叶斯学习的dropoout方法变体
Dropout is used as a practical tool to obtain uncertainty estimates in large vision models and reinforcement learning (RL) tasks. But to obtain well-calibrated uncertainty estimates, a grid-search over the dropout probabilities is necessary— a prohibitive operation with large models, and an impossible one with RL.
We propose a new dropout variant which gives improved performance and better
calibrated uncertainties. Relying on recent developments in Bayesian deep learning,
we use a continuous relaxation of dropout’s discrete masks.
Together with a principled optimisation objective, this allows for automatic tuning of the dropout
probability in large models, and as a result faster experimentation cycles.
In RL this allows the agent to adapt its uncertainty dynamically as more data is observed.
【Concrete|Concrete dropout】We analyse the proposed variant extensively on a range of tasks, and give insights into common practice in the field where larger dropout probabilities are often used in deeper model layers.
推荐阅读
- 《真与假的困惑》???|《真与假的困惑》??? ——致良知是一种伟大的力量
- 午门传说
- 有一种爱叫那一滴眼泪
- 有一种成功,叫“我有时间陪家人”
- 有一种美,叫难以开口!
- 低调做人是一种智慧
- 有一种情
- 安意如(有一种本事把残缺活成了美)
- 极简主义|极简主义 简记
- 吸引力,一种奖赏