Basics of Trust Region Controllers in Xopt¶

Trust Region Bayesian Optimization (TuRBO) is an advanced optimization algorithm designed for solving high-dimensional black-box optimization problems. It combines the strengths of Bayesian Optimization (BO) with trust region methods to improve scalability and efficiency.

Key Features:¶

Trust Regions:
- TuRBO uses local trust regions to focus the search in promising areas of the parameter space.
- Each trust region is a bounded subspace where the optimization is performed, and its size is dynamically adjusted based on the success of the optimization.
Bayesian Surrogate Model:
- A Gaussian Process (GP) or other surrogate models are used to approximate the objective function.
- This surrogate model is used to predict the objective function and guide the search as well as define the size of the trust region.
Adaptivity:
- The algorithm adapts the size of the trust region based on the success or failure of the optimization steps. If the optimization within a trust region is successful, the region expands; otherwise, it shrinks.

Advantages:¶

Scales better to high-dimensional problems compared to standard Bayesian Optimization.
Efficiently balances exploration and exploitation within trust regions.

Disadvantages:¶

Severely restricts exploration of the parameter space potentially leading to convergence to local minima, thus making it sensitive to initial sampling points.
Introduces additional algorithm hyperparameters which can cause issues.
May struggle with noisy objective functions or discontinuous landscapes.

Defining a TuRBO Controller¶

Currently, Xopt supports 3 different TuRBO controller types, the most basic of which is the OptimizeTurboController. To create this controller we need to define our optimization problem and some data.

In [1]:

Copied!





import numpy as np
from xopt import VOCS
import pandas as pd


# create evaluation function
def sphere_function(inputs):
    """
    2D Sphere objective function.
    Compatible with Xopt.
    """
    x, y = inputs["x"], inputs["y"]
    return {"f": np.sum(np.square(np.stack([x, y], axis=-1)), axis=-1)}


# create a VOCS object
vocs = VOCS(
    variables={"x": {-5, 5}, "y": {-5, 5}},
    objectives={"f": "MINIMIZE"},
)

# random sample 10 points
x0 = vocs.random_inputs(10)

# evaluate the function at the random points
f = []
for i in range(len(x0)):
    f += [sphere_function(x0[i]) | x0[i]]

# print the results
data = pd.DataFrame(f)
data
import numpy as np
from xopt import VOCS
import pandas as pd


# create evaluation function
def sphere_function(inputs):
    """
    2D Sphere objective function.
    Compatible with Xopt.
    """
    x, y = inputs["x"], inputs["y"]
    return {"f": np.sum(np.square(np.stack([x, y], axis=-1)), axis=-1)}


# create a VOCS object
vocs = VOCS(
    variables={"x": {-5, 5}, "y": {-5, 5}},
    objectives={"f": "MINIMIZE"},
)

# random sample 10 points
x0 = vocs.random_inputs(10)

# evaluate the function at the random points
f = []
for i in range(len(x0)):
    f += [sphere_function(x0[i]) | x0[i]]

# print the results
data = pd.DataFrame(f)
data

Out[1]:

	f	x	y
0	9.829950	2.423550	1.989059
1	13.616082	2.059389	3.061862
2	8.770805	-2.344175	-1.809876
3	17.541554	2.991614	2.931177
4	3.769704	-1.232220	1.500446
5	12.537187	1.114501	3.360815
6	15.491987	-3.458033	1.879892
7	7.520841	-2.723567	0.320972
8	14.425536	3.221337	2.012094
9	2.243666	-0.621442	1.362893

Create the ExpectedImprovementGenerator and train the GP model¶

Here we create the ExpectedImprovementGenerator, add data to the generator, and train the model from the data.

In [2]:

Copied!





from xopt.generators.bayesian import ExpectedImprovementGenerator

generator = ExpectedImprovementGenerator(vocs=vocs)  # create the generator
generator.gp_constructor.use_low_noise_prior = True
generator.add_data(data)  # add the data to the generator
generator.train_model()  # train the model
from xopt.generators.bayesian import ExpectedImprovementGenerator

generator = ExpectedImprovementGenerator(vocs=vocs)  # create the generator
generator.gp_constructor.use_low_noise_prior = True
generator.add_data(data)  # add the data to the generator
generator.train_model()  # train the model

Out[2]:

ModelListGP(
  (models): ModuleList(
    (0): SingleTaskGP(
      (likelihood): GaussianLikelihood(
        (noise_covar): HomoskedasticNoise(
          (noise_prior): GammaPrior()
          (raw_noise_constraint): GreaterThan(1.000E-04)
        )
      )
      (mean_module): ConstantMean()
      (covar_module): RBFKernel(
        (lengthscale_prior): LogNormalPrior()
        (raw_lengthscale_constraint): GreaterThan(2.500E-02)
      )
      (outcome_transform): Standardize()
      (input_transform): Normalize()
    )
  )
  (likelihood): LikelihoodList(
    (likelihoods): ModuleList(
      (0): GaussianLikelihood(
        (noise_covar): HomoskedasticNoise(
          (noise_prior): GammaPrior()
          (raw_noise_constraint): GreaterThan(1.000E-04)
        )
      )
    )
  )
)

Create the Optimize Turbo Controller¶

Here we create the controller and view the different parameters with their descriptions.

In [3]:

Copied!





from xopt.generators.bayesian.turbo import OptimizeTurboController

turbo_controller = OptimizeTurboController(vocs=vocs)

print(turbo_controller.__doc__)
print("-" * 20)

# examine the attributes of the controller
for field_name, field in turbo_controller.model_fields.items():
    print(f"Field: {field_name}")
    print(f"  Description: {field.description}")
    print(f"  Type: {field.annotation}")
    print(f"  Default: {field.default}")
    print(f"  Value: {getattr(turbo_controller, field_name)}")
    print("-" * 20)
from xopt.generators.bayesian.turbo import OptimizeTurboController

turbo_controller = OptimizeTurboController(vocs=vocs)

print(turbo_controller.__doc__)
print("-" * 20)

# examine the attributes of the controller
for field_name, field in turbo_controller.model_fields.items():
    print(f"Field: {field_name}")
    print(f"  Description: {field.description}")
    print(f"  Type: {field.annotation}")
    print(f"  Default: {field.default}")
    print(f"  Value: {getattr(turbo_controller, field_name)}")
    print("-" * 20)

Turbo controller for optimization tasks.

Attributes:
-----------
name : str
    The name of the controller.
best_value : Optional[float]
    The best value found so far.

Methods:
--------
vocs_validation(cls, info)
    Validate the VOCS for the controller.
minimize(self) -> bool
    Check if the objective is to minimize.
_set_best_point_value(self, data)
    Set the best point value based on the data.
update_state(self, generator, previous_batch_size: int = 1) -> None
    Update the state of the controller.

--------------------
Field: vocs
  Description: VOCS object
  Type: <class 'xopt.vocs.VOCS'>
  Default: PydanticUndefined
  Value: variables={'x': [-5.0, 5.0], 'y': [-5.0, 5.0]} constraints={} objectives={'f': 'MINIMIZE'} constants={} observables=[]
--------------------
Field: dim
  Description: number of dimensions in the optimization problem
  Type: <class 'int'>
  Default: None
  Value: 2
--------------------
Field: batch_size
  Description: number of trust regions to use
  Type: <class 'int'>
  Default: 1
  Value: 1
--------------------
Field: length
  Description: base length of trust region
  Type: <class 'float'>
  Default: 0.25
  Value: 0.25
--------------------
Field: length_min
  Description: minimum base length of trust region
  Type: <class 'float'>
  Default: 0.0078125
  Value: 0.0078125
--------------------
Field: length_max
  Description: maximum base length of trust region
  Type: <class 'float'>
  Default: 2.0
  Value: 2.0
--------------------
Field: failure_counter
  Description: number of failures since reset
  Type: <class 'int'>
  Default: 0
  Value: 0
--------------------
Field: failure_tolerance
  Description: number of failures to trigger a trust region expansion
  Type: <class 'int'>
  Default: None
  Value: 2
--------------------
Field: success_counter
  Description: number of successes since reset
  Type: <class 'int'>
  Default: 0
  Value: 0
--------------------
Field: success_tolerance
  Description: number of successes to trigger a trust region contraction
  Type: <class 'int'>
  Default: None
  Value: 2
--------------------
Field: center_x
  Description: center point of trust region
  Type: typing.Optional[typing.Dict[str, float]]
  Default: None
  Value: None
--------------------
Field: scale_factor
  Description: multiplier to increase or decrease trust region
  Type: <class 'float'>
  Default: 2.0
  Value: 2.0
--------------------
Field: restrict_model_data
  Description: flag to restrict model data to within the trust region
  Type: typing.Optional[bool]
  Default: True
  Value: True
--------------------
Field: name
  Description: name of the Turbo controller
  Type: <class 'str'>
  Default: OptimizeTurboController
  Value: OptimizeTurboController
--------------------
Field: best_value
  Description: best objective value found so far
  Type: typing.Optional[float]
  Default: None
  Value: None
--------------------

/tmp/ipykernel_6697/1076008262.py:9: PydanticDeprecatedSince211: Accessing the 'model_fields' attribute on the instance is deprecated. Instead, you should access this attribute from the model class. Deprecated in Pydantic V2.11 to be removed in V3.0.
  for field_name, field in turbo_controller.model_fields.items():

Getting the Trust Region¶

Here we get the current trust region

In [4]:

Copied!





trust_region = turbo_controller.get_trust_region(
    generator=generator
)  # get the trust region of the model
print(f"Trust Region: {trust_region}")
trust_region = turbo_controller.get_trust_region(
    generator=generator
)  # get the trust region of the model
print(f"Trust Region: {trust_region}")

Trust Region: tensor([[-5., -5.],
        [ 5.,  5.]], dtype=torch.float64)

Update the trust region¶

Add another data point to the generator (as if we performed one optimization step) and update the turbo controller. We will add a point that improves over the best function value measured so far so this measurement will count as a success.

In [5]:

Copied!

# add a new point to the generator
new_point = pd.DataFrame({"x": [0.0], "y": [0.0], "f": [0.0]})
generator.add_data(new_point)  # add the new point to the generator
# add a new point to the generator
new_point = pd.DataFrame({"x": [0.0], "y": [0.0], "f": [0.0]})
generator.add_data(new_point)  # add the new point to the generator

In [6]:

Copied!





generator.train_model()  # train the model again

# update the TuRBO controller
turbo_controller.update_state(generator)

# get the new trust region
trust_region = turbo_controller.get_trust_region(
    generator=generator
)  # get the trust region of the model
print(f"New Trust Region: {trust_region}")

# get the number of successes and failures
print(f"Number of successes: {turbo_controller.success_counter}")
print(f"Number of failures: {turbo_controller.failure_counter}")

# get the base length scale of the trust region
print(f"Base length scale: {turbo_controller.length}")
generator.train_model()  # train the model again

# update the TuRBO controller
turbo_controller.update_state(generator)

# get the new trust region
trust_region = turbo_controller.get_trust_region(
    generator=generator
)  # get the trust region of the model
print(f"New Trust Region: {trust_region}")

# get the number of successes and failures
print(f"Number of successes: {turbo_controller.success_counter}")
print(f"Number of failures: {turbo_controller.failure_counter}")

# get the base length scale of the trust region
print(f"Base length scale: {turbo_controller.length}")

New Trust Region: tensor([[1.2600, 0.6462],
        [3.5871, 3.3319]], dtype=torch.float64)
Number of successes: 1
Number of failures: 0
Base length scale: 0.25

In [ ]: