oneflow.optim.lr_scheduler.PolynomialLR

class oneflow.optim.lr_scheduler.PolynomialLR(optimizer, decay_batch: int, end_learning_rate: float = 0.0001, power: float = 1.0, cycle: bool = False, last_step: int = - 1, verbose: bool = False)

This operator creates a polynomial decayed learning rate scheduler. The learning rate will be updated as follows:

If cycle is True, the equation is:

\[\begin{split}\begin{aligned} & decay\_batch = decay\_batch*ceil(\frac{current\_batch}{decay\_batch}) \\ & learning\_rate = (base\_lr-end\_lr)*(1-\frac{current\_batch}{decay\_batch})^{power}+end\_lr \end{aligned}\end{split}\]

If cycle is False, the equation is:

\[\begin{split}\begin{aligned} & current\_batch = min(decay\_batch, current\_batch) \\ & learning\_rate = (base\_lr-end\_lr)*(1-\frac{current\_batch}{decay\_batch})^{power}+end\_lr \end{aligned}\end{split}\]
Parameters
  • optimizer (Optimizer) – Wrapper optimizer.

  • decay_batch (int) – The decayed steps.

  • end_learning_rate (float, optional) – The final learning rate. Defaults to 0.0001.

  • power (float, optional) – The power of polynomial. Defaults to 1.0.

  • cycle (bool, optional) – If cycle is True, the scheduler will decay the learning rate every decay steps. Defaults to False.

For example:

import oneflow as flow

...
polynomial_scheduler = flow.optim.lr_scheduler.PolynomialLR(
    optimizer, decay_batch=5, end_learning_rate=0.00001, power=2
    )

for epoch in range(num_epoch):
    train(...)
    polynomial_scheduler.step()
__init__(optimizer, decay_batch: int, end_learning_rate: float = 0.0001, power: float = 1.0, cycle: bool = False, last_step: int = - 1, verbose: bool = False)

Initialize self. See help(type(self)) for accurate signature.

Methods

__delattr__(name, /)

Implement delattr(self, name).

__dir__()

Default dir() implementation.

__eq__(value, /)

Return self==value.

__format__(format_spec, /)

Default object formatter.

__ge__(value, /)

Return self>=value.

__getattribute__(name, /)

Return getattr(self, name).

__gt__(value, /)

Return self>value.

__hash__()

Return hash(self).

__init__(optimizer, decay_batch[, …])

Initialize self.

__init_subclass__

This method is called when a class is subclassed.

__le__(value, /)

Return self<=value.

__lt__(value, /)

Return self<value.

__ne__(value, /)

Return self!=value.

__new__(**kwargs)

Create and return a new object.

__reduce__()

Helper for pickle.

__reduce_ex__(protocol, /)

Helper for pickle.

__repr__()

Return repr(self).

__setattr__(name, value, /)

Implement setattr(self, name, value).

__sizeof__()

Size of object in memory, in bytes.

__str__()

Return str(self).

__subclasshook__

Abstract classes can override this to customize issubclass().

_generate_conf_for_graph(lr_conf)

_init_base_lrs()

get_last_lr()

Return last computed learning rate by current scheduler.

get_lr(base_lr, step)

Compute learning rate using chainable form of the scheduler

load_state_dict(state_dict)

Load the schedulers state.

print_lr(group, lr)

Display the current learning rate.

state_dict()

Return the state of the scheduler as a dict.

step()

update_lrs(lrs)