oneflow.optim.Optimizer.add_param_group¶
-
Optimizer.
add_param_group
(param_group) → None¶ Add a param group to the
Optimizer
s param_groups. This can be useful when fine tuning a pre-trained network as frozen layers can be made trainable and added to theOptimizer
as training progresses.- Parameters
param_group (dict) – Specifies what Tensors should be optimized along with group specific optimization options.
Example:
>>> import oneflow >>> import oneflow.optim as optim >>> w1 = oneflow.ones(3, 3) >>> w1.requires_grad = True >>> w2 = oneflow.ones(3, 3) >>> w2.requires_grad = True >>> o = optim.SGD([w1]) >>> o.param_groups[0] {'lr': 0.001, 'momentum': 0.0, 'dampening': 0.0, 'weight_decay': 0.0, 'nesterov': False, 'maximize': False, 'params': [tensor([[1., 1., 1.], [1., 1., 1.], [1., 1., 1.]], dtype=oneflow.float32, requires_grad=True)]} >>> o.add_param_group({'params': w2}) >>> o.param_groups[1] {'lr': 0.001, 'momentum': 0.0, 'dampening': 0.0, 'weight_decay': 0.0, 'nesterov': False, 'maximize': False, 'params': [tensor([[1., 1., 1.], [1., 1., 1.], [1., 1., 1.]], dtype=oneflow.float32, requires_grad=True)]}