Constrained optimization

Constrained Bayesian Optimization with EI and UCB¶

In this tutorial we demonstrate the use of Xopt to perform Bayesian Optimization on a simple test problem subject to a single constraint. We will compare two acquisition functions:

Expected Improvement (EI) - balances exploitation and exploration by targeting areas with high probability of improvement
Upper Confidence Bound (UCB) - uses an explicit $\beta$ parameter to balance exploration and exploitation. However, it requires special consideration when using with constraints

Define the test problem¶

Here we define a simple optimization problem, where we attempt to minimize the sin function in the domian [0,2*pi], subject to a cos constraining function.

In [1]:

Copied!





from xopt.evaluator import Evaluator
from xopt.generators.bayesian import (
    ExpectedImprovementGenerator,
    UpperConfidenceBoundGenerator,
)
from xopt import Xopt
from xopt.vocs import VOCS

import time
import math
import numpy as np
import matplotlib.pyplot as plt

# Ignore all warnings
import warnings

warnings.filterwarnings("ignore")

# define fixed seed
np.random.seed(42)

# define variables, function objective and constraining function
vocs = VOCS(
    variables={"x": [0, 2 * math.pi]},
    objectives={"f": "MINIMIZE"},
    constraints={"c": ["LESS_THAN", 0]},
)
from xopt.evaluator import Evaluator
from xopt.generators.bayesian import (
    ExpectedImprovementGenerator,
    UpperConfidenceBoundGenerator,
)
from xopt import Xopt
from xopt.vocs import VOCS

import time
import math
import numpy as np
import matplotlib.pyplot as plt

# Ignore all warnings
import warnings

warnings.filterwarnings("ignore")

# define fixed seed
np.random.seed(42)

# define variables, function objective and constraining function
vocs = VOCS(
    variables={"x": [0, 2 * math.pi]},
    objectives={"f": "MINIMIZE"},
    constraints={"c": ["LESS_THAN", 0]},
)

In [2]:

Copied!

# define a test function to optimize
def test_function(input_dict):
    return {"f": np.sin(input_dict["x"]), "c": np.cos(input_dict["x"] + 0.5)}
# define a test function to optimize
def test_function(input_dict):
    return {"f": np.sin(input_dict["x"]), "c": np.cos(input_dict["x"] + 0.5)}

Constrained Bayesian Optimization with Expected Improvement (EI)¶

Create the evaluator to evaluate our test function and create a generator that uses the Expected Improvement acquisition function to perform Bayesian Optimization. Note that because we are optimizing a problem with no noise we set use_low_noise_prior=True in the GP model constructor.

In [3]:

Copied!





evaluator = Evaluator(function=test_function)
generator_ei = ExpectedImprovementGenerator(vocs=vocs)
generator_ei.gp_constructor.use_low_noise_prior = True
X_ei = Xopt(evaluator=evaluator, generator=generator_ei, vocs=vocs)
evaluator = Evaluator(function=test_function)
generator_ei = ExpectedImprovementGenerator(vocs=vocs)
generator_ei.gp_constructor.use_low_noise_prior = True
X_ei = Xopt(evaluator=evaluator, generator=generator_ei, vocs=vocs)

Generate and evaluate initial points for EI¶

To begin optimization, we must generate some random initial data points. The first call to X.step() will generate and evaluate a number of randomly points specified by the generator. Note that if we add data to xopt before calling X.step() by assigning the data to X.data, calls to X.step() will ignore the random generation and proceed to generating points via Bayesian optimization.

In [4]:

Copied!





# call X.random_evaluate(n_samples) to generate + evaluate initial points
X_ei.random_evaluate(n_samples=3)

# inspect the gathered data
print("Initial data for EI optimization:")
print(X_ei.data)
# call X.random_evaluate(n_samples) to generate + evaluate initial points
X_ei.random_evaluate(n_samples=3)

# inspect the gathered data
print("Initial data for EI optimization:")
print(X_ei.data)

Initial data for EI optimization:
          x         f         c  xopt_runtime  xopt_error
0  3.929880 -0.709147 -0.278766      0.000013       False
1  0.309671  0.304745  0.689737      0.000002       False
2  1.683932  0.993607 -0.575435      0.000001       False

Do Bayesian optimization steps with EI¶

To perform optimization we simply call X.step() in a loop. This allows us to do intermediate tasks in between optimization steps, such as examining the model and acquisition function at each step (as we demonstrate here).

In [5]:

Copied!





n_steps = 5

# test points for plotting
test_x = np.linspace(*X_ei.vocs.bounds.flatten(), 50)

print("=== Expected Improvement Optimization ===")
for i in range(n_steps):
    print(f"EI Step {i + 1}")
    start = time.perf_counter()

    # train model and visualize
    model = X_ei.generator.train_model()
    fig, ax = X_ei.generator.visualize_model(n_grid=100)
    fig.suptitle(f"Expected Improvement - Step {i + 1}")

    # add ground truth functions to plots
    out = test_function({"x": test_x})
    ax[0, 0].plot(test_x, out["f"], "C0-.", label="True objective", linewidth=2)
    ax[1, 0].plot(test_x, out["c"], "C2-.", label="True constraint", linewidth=2)
    ax[0, 0].legend()
    ax[1, 0].legend()

    plt.show()
    print(f"Time: {time.perf_counter() - start:.3f}s")

    # do the optimization step
    X_ei.step()
    print(f"Current best: f = {X_ei.vocs.select_best(X_ei.data)[1].item():.4f}")
    print("-" * 40)
n_steps = 5

# test points for plotting
test_x = np.linspace(*X_ei.vocs.bounds.flatten(), 50)

print("=== Expected Improvement Optimization ===")
for i in range(n_steps):
    print(f"EI Step {i + 1}")
    start = time.perf_counter()

    # train model and visualize
    model = X_ei.generator.train_model()
    fig, ax = X_ei.generator.visualize_model(n_grid=100)
    fig.suptitle(f"Expected Improvement - Step {i + 1}")

    # add ground truth functions to plots
    out = test_function({"x": test_x})
    ax[0, 0].plot(test_x, out["f"], "C0-.", label="True objective", linewidth=2)
    ax[1, 0].plot(test_x, out["c"], "C2-.", label="True constraint", linewidth=2)
    ax[0, 0].legend()
    ax[1, 0].legend()

    plt.show()
    print(f"Time: {time.perf_counter() - start:.3f}s")

    # do the optimization step
    X_ei.step()
    print(f"Current best: f = {X_ei.vocs.select_best(X_ei.data)[1].item():.4f}")
    print("-" * 40)

=== Expected Improvement Optimization ===
EI Step 1

No description has been provided for this image

Time: 0.751s

Current best: f = -0.7091
----------------------------------------
EI Step 2

Time: 0.432s

Current best: f = -0.7091
----------------------------------------
EI Step 3

Time: 0.435s

Current best: f = -0.8685
----------------------------------------
EI Step 4

Time: 0.624s

Current best: f = -0.8750
----------------------------------------
EI Step 5

Time: 0.419s

Current best: f = -0.8762
----------------------------------------

In [6]:

Copied!





# access the collected data from EI optimization
print("Final EI optimization results:")
print(X_ei.data)
print(f"EI Best valid solution: {X_ei.vocs.select_best(X_ei.data)[1].item()}")
# access the collected data from EI optimization
print("Final EI optimization results:")
print(X_ei.data)
print(f"EI Best valid solution: {X_ei.vocs.select_best(X_ei.data)[1].item()}")

Final EI optimization results:
          x         f         c  xopt_runtime  xopt_error
0  3.929880 -0.709147 -0.278766      0.000013       False
1  0.309671  0.304745  0.689737      0.000002       False
2  1.683932  0.993607 -0.575435      0.000001       False
3  4.394979 -0.950047  0.181578      0.000008       False
4  5.502802 -0.703552  0.960949      0.000008       False
5  4.193807 -0.868523 -0.018581      0.000008       False
6  4.207058 -0.875014 -0.005331      0.000008       False
7  4.209597 -0.876241 -0.002792      0.000008       False
EI Best valid solution: -0.8762405433246848

Constrained Bayesian Optimization with Upper Confidence Bound (UCB)¶

The Upper Confidence Bound acquisition function allows a user to explicitly specify the balance between exploration and exploitation. However, for constrained optimization, there is an important technical requirement:

Constraint handling requires positive acquisition values: When constraints are present, the acquisition function is weighted by the probability of feasibility, this will only work with strictly positive acquisition function values!.
The shift parameter ensures positivity: Adding a positive shift ensures the UCB acquisition function is strictly positive, which is required for proper constraint weighting. Note that this addition will not change the location of the acquisition function maximum.

In [7]:

Copied!





# Create UCB generator with shift parameter for constrained optimization
# The shift parameter ensures the acquisition function is strictly positive
# This is required because constrained optimization weights the acquisition
# function by the probability of feasibility: α_constrained = α_unconstrained × P(feasible)
generator_ucb = UpperConfidenceBoundGenerator(vocs=vocs, shift=2.0)
generator_ucb.gp_constructor.use_low_noise_prior = True

# Create new Xopt object for UCB
X_ucb = Xopt(evaluator=evaluator, generator=generator_ucb, vocs=vocs)

print("UCB Generator configuration:")
print(f"Shift parameter: {generator_ucb.shift}")
print("This shift ensures UCB values are strictly positive for constraint weighting")
# Create UCB generator with shift parameter for constrained optimization
# The shift parameter ensures the acquisition function is strictly positive
# This is required because constrained optimization weights the acquisition
# function by the probability of feasibility: α_constrained = α_unconstrained × P(feasible)
generator_ucb = UpperConfidenceBoundGenerator(vocs=vocs, shift=2.0)
generator_ucb.gp_constructor.use_low_noise_prior = True

# Create new Xopt object for UCB
X_ucb = Xopt(evaluator=evaluator, generator=generator_ucb, vocs=vocs)

print("UCB Generator configuration:")
print(f"Shift parameter: {generator_ucb.shift}")
print("This shift ensures UCB values are strictly positive for constraint weighting")

UCB Generator configuration:
Shift parameter: 2.0
This shift ensures UCB values are strictly positive for constraint weighting

In [8]:

Copied!

# Generate initial points for UCB optimization
X_ucb.random_evaluate(n_samples=3)

print("Initial data for UCB optimization:")
print(X_ucb.data)
# Generate initial points for UCB optimization
X_ucb.random_evaluate(n_samples=3)

print("Initial data for UCB optimization:")
print(X_ucb.data)

Initial data for UCB optimization:
          x         f         c  xopt_runtime  xopt_error
0  2.521703  0.580945 -0.992822  6.482000e-06       False
1  5.302891 -0.830661  0.886859  1.302000e-06       False
2  5.303043 -0.830577  0.886929  8.120001e-07       False

In [9]:

Copied!





print("=== Upper Confidence Bound Optimization ===")
for i in range(n_steps):
    print(f"UCB Step {i + 1}")
    start = time.perf_counter()

    # train model and visualize
    model = X_ucb.generator.train_model()
    fig, ax = X_ucb.generator.visualize_model(n_grid=100)
    fig.suptitle(f"Upper Confidence Bound (shift={generator_ucb.shift}) - Step {i + 1}")

    # add ground truth functions to plots
    out = test_function({"x": test_x})
    ax[0, 0].plot(test_x, out["f"], "C0-.", label="True objective", linewidth=2)
    ax[1, 0].plot(test_x, out["c"], "C2-.", label="True constraint", linewidth=2)
    ax[0, 0].legend()
    ax[1, 0].legend()

    plt.show()
    print(f"Time: {time.perf_counter() - start:.3f}s")

    # do the optimization step
    X_ucb.step()
    print(f"Current best: f = {X_ucb.vocs.select_best(X_ucb.data)[1].item():.4f}")
    print("-" * 40)
print("=== Upper Confidence Bound Optimization ===")
for i in range(n_steps):
    print(f"UCB Step {i + 1}")
    start = time.perf_counter()

    # train model and visualize
    model = X_ucb.generator.train_model()
    fig, ax = X_ucb.generator.visualize_model(n_grid=100)
    fig.suptitle(f"Upper Confidence Bound (shift={generator_ucb.shift}) - Step {i + 1}")

    # add ground truth functions to plots
    out = test_function({"x": test_x})
    ax[0, 0].plot(test_x, out["f"], "C0-.", label="True objective", linewidth=2)
    ax[1, 0].plot(test_x, out["c"], "C2-.", label="True constraint", linewidth=2)
    ax[0, 0].legend()
    ax[1, 0].legend()

    plt.show()
    print(f"Time: {time.perf_counter() - start:.3f}s")

    # do the optimization step
    X_ucb.step()
    print(f"Current best: f = {X_ucb.vocs.select_best(X_ucb.data)[1].item():.4f}")
    print("-" * 40)

=== Upper Confidence Bound Optimization ===
UCB Step 1

Time: 0.419s

Current best: f = 0.5809
----------------------------------------
UCB Step 2

Time: 0.415s

Current best: f = -0.1238
----------------------------------------
UCB Step 3

Time: 0.422s

Current best: f = -0.7357
----------------------------------------
UCB Step 4

Time: 0.418s

Current best: f = -0.8623
----------------------------------------
UCB Step 5

Time: 0.440s

Current best: f = -0.8678
----------------------------------------

In [10]:

Copied!





# access the collected data from UCB optimization
print("Final UCB optimization results:")
print(X_ucb.data)
print(f"UCB Best valid solution: {X_ucb.vocs.select_best(X_ucb.data)[1].item()}")
# access the collected data from UCB optimization
print("Final UCB optimization results:")
print(X_ucb.data)
print(f"UCB Best valid solution: {X_ucb.vocs.select_best(X_ucb.data)[1].item()}")

Final UCB optimization results:
          x         f         c  xopt_runtime  xopt_error
0  2.521703  0.580945 -0.992822  6.482000e-06       False
1  5.302891 -0.830661  0.886859  1.302000e-06       False
2  5.303043 -0.830577  0.886929  8.120001e-07       False
3  1.666087  0.995463 -0.560749  1.158200e-05       False
4  3.265683 -0.123772 -0.811495  6.722000e-06       False
5  3.968268 -0.735684 -0.241703  7.534000e-06       False
6  4.181334 -0.862273 -0.031050  7.003000e-06       False
7  4.192396 -0.867823 -0.019991  8.576000e-06       False
UCB Best valid solution: -0.8678228500728544

Key Takeaways¶

Expected Improvement (EI):

Natural balance between exploration and exploitation
Works well out-of-the-box for both maximization and minimization
Always positive, making it naturally compatible with constraints
Tends to be more exploitative near the current best

Upper Confidence Bound (UCB) for Constrained Optimization:

Requires shift parameter for constraints: UCB can be negative, but constrained optimization requires positive acquisition values
Technical requirement: α_constrained(x) = α_unconstrained(x) × P(feasible|x) needs α_unconstrained(x) > 0
Shift > 0: Ensures UCB acquisition function is strictly positive

With Constraints:

Both acquisition functions handle constraints by incorporating constraint predictions
The visualization shows both the objective model and constraint model
Acquisition function values are weighted by probability of feasibility