oneflow.optim.Optimizer.add_param_group

Optimizer.add_param_group(param_group)None

Add a param group to the Optimizer s param_groups. This can be useful when fine tuning a pre-trained network as frozen layers can be made trainable and added to the Optimizer as training progresses.

Parameters

param_group (dict) – Specifies what Tensors should be optimized along with group specific optimization options.

Example:

>>> import oneflow
>>> import oneflow.optim as optim
>>> w1 = oneflow.ones(3, 3)
>>> w1.requires_grad = True
>>> w2 = oneflow.ones(3, 3)
>>> w2.requires_grad = True
>>> o = optim.SGD([w1])
>>> o.param_groups[0]
{'lr': 0.001, 'momentum': 0.0, 'dampening': 0.0, 'weight_decay': 0.0, 'nesterov': False, 'maximize': False, 'params': [tensor([[1., 1., 1.],
        [1., 1., 1.],
        [1., 1., 1.]], dtype=oneflow.float32, requires_grad=True)]}
>>> o.add_param_group({'params': w2})
>>> o.param_groups[1]
{'lr': 0.001, 'momentum': 0.0, 'dampening': 0.0, 'weight_decay': 0.0, 'nesterov': False, 'maximize': False, 'params': [tensor([[1., 1., 1.],
        [1., 1., 1.],
        [1., 1., 1.]], dtype=oneflow.float32, requires_grad=True)]}