otf.optim.lr_scheduler

Learning-rate schedulers for optimizers.

Provides a small set of scheduler helpers that adjust an optimizer's learning_rate attribute over time. Implementations are lightweight and intended to be used with BaseOptimizer instances in the optim package.

Classes:

Name	Description
DummyLRScheduler	No-op scheduler for testing and compatibility.
ExponentialLR	Multiply an optimizer's learning rate by a constant factor on each
LRScheduler	Abstract base for learning-rate schedulers that modify an optimizer's
MultiStepLR	Reduce learning rate at specified step milestones.

DummyLRScheduler

DummyLRScheduler(*args, **kwargs)

Bases: LRScheduler

No-op scheduler for testing and compatibility.

Initialize a dummy scheduler (accepts arbitrary args).

This scheduler performs no action when step is called.

Source code in src/otf/optim/lr_scheduler.py

def __init__(self, *args, **kwargs):
    """Initialize a dummy scheduler (accepts arbitrary args).

    This scheduler performs no action when `step` is called.
    """
    pass

ExponentialLR

ExponentialLR(
    optimizer: BaseOptimizer, gamma: float = 0.99
)

Bases: LRScheduler

Multiply an optimizer's learning rate by a constant factor on each step() call.

Initialize the exponential scheduler.

Parameters:

Name	Type	Description	Default
optimizer	BaseOptimizer	An instance of BaseOptimizer with a learning_rate attribute.	required
gamma	float	Factor to multiply the learning rate by on each step.	0.99

Source code in src/otf/optim/lr_scheduler.py

def __init__(self, optimizer: BaseOptimizer, gamma: float = 0.99):
    """Initialize the exponential scheduler.

    Parameters
    ----------
    optimizer
        An instance of `BaseOptimizer` with a `learning_rate` attribute.
    gamma
        Factor to multiply the learning rate by on each `step`.
    """
    super().__init__(optimizer)
    self.gamma = gamma

LRScheduler

LRScheduler(optimizer: BaseOptimizer)

Abstract base for learning-rate schedulers that modify an optimizer's learning_rate attribute.

Create a scheduler bound to optimizer.

Parameters:

Name	Type	Description	Default
optimizer	BaseOptimizer	BaseOptimizer instance whose learning_rate attribute will be adjusted.	required

Source code in src/otf/optim/lr_scheduler.py

def __init__(self, optimizer: BaseOptimizer):
    """Create a scheduler bound to `optimizer`.

    Parameters
    ----------
    optimizer
        `BaseOptimizer` instance whose `learning_rate` attribute will be
        adjusted.
    """
    self.optimizer = optimizer

MultiStepLR

MultiStepLR(
    optimizer: BaseOptimizer,
    milestones: list[int] | tuple[int],
    gamma: float = 0.5,
)

Bases: LRScheduler

Reduce learning rate at specified step milestones.

Initialize the multi-step scheduler.

Parameters:

Name	Type	Description	Default
optimizer	BaseOptimizer	An instance of BaseOptimizer with a learning_rate attribute.	required
milestones	list[int] | tuple[int]	For each milestone, update the learning rate after that many calls to step. Specifying the same milestone multiple times multiplies the learning rate repeatedly at that milestone.	required
gamma	float	Factor by which to multiply the learning rate at each milestone.	0.5

Source code in src/otf/optim/lr_scheduler.py

def __init__(
    self,
    optimizer: BaseOptimizer,
    milestones: list[int] | tuple[int],
    gamma: float = 0.5,
):
    """Initialize the multi-step scheduler.

    Parameters
    ----------
    optimizer
        An instance of `BaseOptimizer` with a `learning_rate` attribute.
    milestones
        For each milestone, update the learning rate after that many calls
        to `step`. Specifying the same milestone multiple times multiplies
        the learning rate repeatedly at that milestone.
    gamma
        Factor by which to multiply the learning rate at each milestone.
    """
    super().__init__(optimizer)
    self.milestones = Counter(milestones)
    self.gamma = gamma
    self.steps = 0