Basic example

Building GP Models from Scratch¶

Sometimes it is useful to build GP models outside the context of BO for data visualization and senativity measurements, ie. learned hyperparameters. Here we demonstrate how to build models from data outside of generators.

For this we use the 3D rosenbrock function test function.

In [1]:

# set values if testing
import os

from xopt import Xopt, Evaluator
from xopt.generators import RandomGenerator
from xopt.resources.test_functions.rosenbrock import (
    evaluate_rosenbrock,
    make_rosenbrock_vocs,
)

from xopt.generators.bayesian.visualize import visualize_model
from xopt.generators.bayesian.models.standard import StandardModelConstructor

# Ignore all warnings
import warnings

warnings.filterwarnings("ignore")


SMOKE_TEST = os.environ.get("SMOKE_TEST")
NUM_MC_SAMPLES = 1 if SMOKE_TEST else 128
NUM_RESTARTS = 1 if SMOKE_TEST else 20

# make rosenbrock function vocs in 3D
vocs = make_rosenbrock_vocs(3)

# collect some data using random sampling
evaluator = Evaluator(function=evaluate_rosenbrock)
generator = RandomGenerator(vocs=vocs)
X = Xopt(generator=generator, evaluator=evaluator, vocs=vocs)
X.random_evaluate(15)

# set values if testing import os from xopt import Xopt, Evaluator from xopt.generators import RandomGenerator from xopt.resources.test_functions.rosenbrock import ( evaluate_rosenbrock, make_rosenbrock_vocs, ) from xopt.generators.bayesian.visualize import visualize_model from xopt.generators.bayesian.models.standard import StandardModelConstructor # Ignore all warnings import warnings warnings.filterwarnings("ignore") SMOKE_TEST = os.environ.get("SMOKE_TEST") NUM_MC_SAMPLES = 1 if SMOKE_TEST else 128 NUM_RESTARTS = 1 if SMOKE_TEST else 20 # make rosenbrock function vocs in 3D vocs = make_rosenbrock_vocs(3) # collect some data using random sampling evaluator = Evaluator(function=evaluate_rosenbrock) generator = RandomGenerator(vocs=vocs) X = Xopt(generator=generator, evaluator=evaluator, vocs=vocs) X.random_evaluate(15)

Out[1]:

	x0	x1	x2	y	xopt_runtime	xopt_error
0	1.210178	-0.552150	0.609807	418.452556	0.000010	False
1	-1.085912	-0.231967	0.986210	291.946897	0.000005	False
2	1.871764	-0.182787	1.640982	1619.457275	0.000006	False
3	0.525314	0.046024	0.769531	65.314425	0.000002	False
4	-1.296144	0.181742	-1.155013	371.560959	0.000002	False
5	1.703212	-0.157229	-1.352367	1126.705232	0.000002	False
6	1.765282	0.958056	-0.169222	584.532721	0.000002	False
7	0.488712	-1.243589	-0.322317	574.307086	0.000002	False
8	-1.647637	1.604177	-0.051238	819.566983	0.000002	False
9	-1.021871	1.571100	0.898723	278.548826	0.000002	False
10	0.342705	1.904902	1.018192	1002.199393	0.000002	False
11	-1.879500	-0.688287	-0.780986	1950.094900	0.000002	False
12	0.749052	0.694104	0.767693	10.100724	0.000002	False
13	1.227592	0.032722	1.662598	494.398986	0.000002	False
14	1.288446	-1.474082	-0.650878	1785.892335	0.000002	False

Create GP model based on the data¶

In [2]:

data = X.data

data = X.data

In [3]:

model_constructor = StandardModelConstructor()

# here we build a model from vocs
model = model_constructor.build_model_from_vocs(vocs=vocs, data=data)

# here we build a model from info (more flexible)
model = model_constructor.build_model(
    input_names=["x0", "x1", "x2"], outcome_names=["y"], data=data
)
help(model_constructor.build_model)

model_constructor = StandardModelConstructor() # here we build a model from vocs model = model_constructor.build_model_from_vocs(vocs=vocs, data=data) # here we build a model from info (more flexible) model = model_constructor.build_model( input_names=["x0", "x1", "x2"], outcome_names=["y"], data=data ) help(model_constructor.build_model)

Help on method build_model in module xopt.generators.bayesian.models.standard:

build_model(
    input_names: List[str],
    outcome_names: List[str],
    data: pandas.core.frame.DataFrame,
    input_bounds: Dict[str, List] = None,
    dtype: torch.dtype = torch.float64,
    device: Union[torch.device, str] = 'cpu'
) -> botorch.models.model_list_gp_regression.ModelListGP method of xopt.generators.bayesian.models.standard.StandardModelConstructor instance
    Construct independent models for each objective and constraint.

    Parameters
    ----------
    input_names : List[str]
        Names of input variables.
    outcome_names : List[str]
        Names of outcome variables.
    data : pd.DataFrame
        Data used for training the model.
    input_bounds : Dict[str, List], optional
        Bounds for input variables.
    dtype : torch.dtype, optional
        Data type for the model (default is torch.double).
    device : Union[torch.device, str], optional
        Device on which to perform computations (default is "cpu").

    Returns
    -------
    ModelListGP
        A list of trained botorch models.

Examine GP model hyperparameters¶

Here we look at the GP hyperparameters for the objective function (the first model). Note: the hyperparameters here are in raw_units (due to contraints on parameter values, ie. lengthscales > 0)

In [4]:

objective_model = model.models[vocs.output_names.index("y")]

# print raw hyperparameter values
for name, val in objective_model.named_parameters():
    print(name, val)

objective_model = model.models[vocs.output_names.index("y")] # print raw hyperparameter values for name, val in objective_model.named_parameters(): print(name, val)

likelihood.noise_covar.raw_noise Parameter containing:
tensor([0.1723], dtype=torch.float64, requires_grad=True)
mean_module.raw_constant Parameter containing:
tensor(0.3749, dtype=torch.float64, requires_grad=True)
covar_module.raw_lengthscale Parameter containing:
tensor([[0.1446, 0.4042, 1.5209]], dtype=torch.float64, requires_grad=True)

In [5]:

# print real values - note that these are in normalized coordinates
print("prior mean:", objective_model.mean_module.constant.data)
print("noise:", objective_model.likelihood.noise_covar.noise.data)
print("lengthscales", objective_model.covar_module.lengthscale.data)

# print real values - note that these are in normalized coordinates print("prior mean:", objective_model.mean_module.constant.data) print("noise:", objective_model.likelihood.noise_covar.noise.data) print("lengthscales", objective_model.covar_module.lengthscale.data)

prior mean: tensor(0.3749, dtype=torch.float64)
noise: tensor([0.1723], dtype=torch.float64)
lengthscales tensor([[0.1446, 0.4042, 1.5209]], dtype=torch.float64)

In [6]:

objective_model

objective_model

Out[6]:

SingleTaskGP(
  (likelihood): GaussianLikelihood(
    (noise_covar): HomoskedasticNoise(
      (noise_prior): GammaPrior()
      (raw_noise_constraint): GreaterThan(1.000E-04)
    )
  )
  (mean_module): ConstantMean()
  (covar_module): RBFKernel(
    (lengthscale_prior): LogNormalPrior()
    (raw_lengthscale_constraint): GreaterThan(2.500E-02)
  )
  (outcome_transform): Standardize()
  (input_transform): Normalize()
)

Visualize model predictions¶

In [7]:

fig, ax = visualize_model(
    model, vocs, data, variable_names=["x0", "x1"], reference_point={"x2": 0.0}
)

fig, ax = visualize_model( model, vocs, data, variable_names=["x0", "x1"], reference_point={"x2": 0.0} )