Bayesian generators

xopt.generators.bayesian.bayesian_generator.BayesianGenerator

BayesianGenerator(**kwargs)

Bases: Generator, ABC

Bayesian Generator for Bayesian Optimization.

Attributes:

name : str The name of the Bayesian Generator.

model : Optional[Model] The BoTorch model used by the generator to perform optimization.

n_monte_carlo_samples : int The number of Monte Carlo samples to use in the optimization process.

turbo_controller : SerializeAsAny[Optional[TurboController]] The Turbo Controller for trust-region Bayesian Optimization.

use_cuda : bool A flag to enable or disable CUDA usage if available.

gp_constructor : SerializeAsAny[ModelConstructor] The constructor used to generate the model for Bayesian Optimization.

numerical_optimizer : SerializeAsAny[NumericalOptimizer] The optimizer used to optimize the acquisition function in Bayesian Optimization.

max_travel_distances : Optional[List[float]] The limits for travel distances between points in normalized space.

fixed_features : Optional[Dict[str, float]] The fixed features used in Bayesian Optimization.

computation_time : Optional[pd.DataFrame] A data frame tracking computation time in seconds.

log_transform_acquisition_function: Optional[bool] Flag to determine if final acquisition function value should be log-transformed before optimization.

n_interpolate_samples: Optional[PositiveInt] Number of interpolation points to generate between last observation and next observation, requires n_candidates to be 1.

n_candidates : int The number of candidates to generate in each optimization step.

Methods:

generate(self, n_candidates: int) -> List[Dict]: Generate candidates for Bayesian Optimization.

add_data(self, new_data: pd.DataFrame): Add new data to the generator for Bayesian Optimization.

train_model(self, data: pd.DataFrame = None, update_internal=True) -> Module: Train a Bayesian model for Bayesian Optimization.

propose_candidates(self, model, n_candidates=1) -> Tensor: Propose candidates for Bayesian Optimization.

get_input_data(self, data: pd.DataFrame) -> torch.Tensor: Get input data in torch.Tensor format.

get_acquisition(self, model) -> AcquisitionFunction: Get the acquisition function for Bayesian Optimization.

Source code in xopt/generator.py

def __init__(self, **kwargs):
    """
    Initialize the generator.

    """
    super().__init__(**kwargs)
    logger.info(f"Initialized generator {self.name}")

Attributes

xopt.generators.bayesian.bayesian_generator.BayesianGenerator.model_input_names property

model_input_names

variable names corresponding to trained model

Functions

xopt.generators.bayesian.bayesian_generator.BayesianGenerator.generate

generate(n_candidates)

Generate candidates using Bayesian Optimization.

Parameters:

n_candidates : int The number of candidates to generate in each optimization step.

Returns:

List[Dict] A list of dictionaries containing the generated candidates.

Raises:

NotImplementedError If the number of candidates is greater than 1, and the generator does not support batch candidate generation.

RuntimeError If no data is contained in the generator, the 'add_data' method should be called to add data before generating candidates.

Notes:

This method generates candidates for Bayesian Optimization based on the provided number of candidates. It updates the internal model with the current data and calculates the candidates by optimizing the acquisition function. The method returns the generated candidates in the form of a list of dictionaries.

Source code in xopt/generators/bayesian/bayesian_generator.py

def generate(self, n_candidates: int) -> list[dict]:
    """
    Generate candidates using Bayesian Optimization.

    Parameters:
    -----------
    n_candidates : int
        The number of candidates to generate in each optimization step.

    Returns:
    --------
    List[Dict]
        A list of dictionaries containing the generated candidates.

    Raises:
    -------
    NotImplementedError
        If the number of candidates is greater than 1, and the generator does not
        support batch candidate generation.

    RuntimeError
        If no data is contained in the generator, the 'add_data' method should be
        called to add data before generating candidates.

    Notes:
    ------
    This method generates candidates for Bayesian Optimization based on the
    provided number of candidates. It updates the internal model with the current
    data and calculates the candidates by optimizing the acquisition function.
    The method returns the generated candidates in the form of a list of dictionaries.
    """

    self.n_candidates = n_candidates
    if n_candidates > 1 and not self.supports_batch_generation:
        raise NotImplementedError(
            "This Bayesian algorithm does not currently support parallel candidate "
            "generation"
        )

    # if no data exists raise error
    if self.data is None:
        raise RuntimeError(
            "no data contained in generator, call `add_data` "
            "method to add data, see also `Xopt.random_evaluate()`"
        )

    else:
        # dict to track runtimes
        timing_results = {}

        # update internal model with internal data
        start_time = time.perf_counter()
        model = self.train_model(self.get_training_data(self.data))
        timing_results["training"] = time.perf_counter() - start_time

        # propose candidates given model
        start_time = time.perf_counter()
        candidates = self.propose_candidates(model, n_candidates=n_candidates)
        timing_results["acquisition_optimization"] = (
            time.perf_counter() - start_time
        )

        # post process candidates
        result = self._process_candidates(candidates)

        # append timing results to dataframe (if it exists)
        if self.computation_time is not None:
            self.computation_time = pd.concat(
                (
                    self.computation_time,
                    pd.DataFrame(timing_results, index=[0]),
                ),
                ignore_index=True,
            )
        else:
            self.computation_time = pd.DataFrame(timing_results, index=[0])

        if self.n_interpolate_points is not None:
            if self.n_candidates > 1:
                raise RuntimeError(
                    "cannot generate interpolated points for "
                    "multiple candidate generation"
                )
            else:
                assert len(result) == 1
                result = interpolate_points(
                    pd.concat(
                        (self.data.iloc[-1:][self.vocs.variable_names], result),
                        axis=0,
                        ignore_index=True,
                    ),
                    num_points=self.n_interpolate_points,
                )

        return result.to_dict("records")

xopt.generators.bayesian.bayesian_generator.BayesianGenerator.get_acquisition

get_acquisition(model)

Define the acquisition function based on the given GP model.

Parameters:

model : Model The BoTorch model to be used for generating the acquisition function.

Returns:

acqusition_function : AcqusitionFunction

Raises:

ValueError If the provided 'model' is None. A valid model is required to create the acquisition function.

Source code in xopt/generators/bayesian/bayesian_generator.py

def get_acquisition(self, model):
    """
    Define the acquisition function based on the given GP model.

    Parameters:
    -----------
    model : Model
        The BoTorch model to be used for generating the acquisition function.

    Returns:
    --------
    acqusition_function : AcqusitionFunction

    Raises:
    -------
    ValueError
        If the provided 'model' is None. A valid model is required to create the
        acquisition function.
    """
    if model is None:
        raise ValueError("model cannot be None")

    # get base acquisition function
    acq = self._get_acquisition(model)

    # apply constraints if specified in vocs
    # TODO: replace with direct constrainted acquisition function calls
    # see SampleReducingMCAcquisitionFunction in botorch for rationale
    if len(self.vocs.constraints):
        try:
            sampler = acq.sampler
        except AttributeError:
            sampler = self._get_sampler(model)

        acq = ConstrainedMCAcquisitionFunction(
            model, acq, self._get_constraint_callables(), sampler=sampler
        )

    # apply fixed features if specified in the generator
    if self.fixed_features is not None:
        # get input dim
        dim = len(self.model_input_names)
        columns = []
        values = []
        for name, value in self.fixed_features.items():
            columns += [self.model_input_names.index(name)]
            values += [value]

        acq = FixedFeatureAcquisitionFunction(
            acq_function=acq, d=dim, columns=columns, values=values
        )

    if self.log_transform_acquisition_function:
        acq = LogAcquisitionFunction(acq)

    return acq

xopt.generators.bayesian.bayesian_generator.BayesianGenerator.get_input_data

get_input_data(data)

Convert input data to a torch tensor.

Parameters:

data : pd.DataFrame The input data in the form of a pandas DataFrame.

Returns:

torch.Tensor A torch tensor containing the input data.

Notes:

This method takes a pandas DataFrame as input data and converts it into a torch tensor. It specifically selects columns corresponding to the model's input names (variables), and the resulting tensor is configured with the data type and device settings from the generator.

Source code in xopt/generators/bayesian/bayesian_generator.py

def get_input_data(self, data: pd.DataFrame) -> torch.Tensor:
    """
    Convert input data to a torch tensor.

    Parameters:
    -----------
    data : pd.DataFrame
        The input data in the form of a pandas DataFrame.

    Returns:
    --------
    torch.Tensor
        A torch tensor containing the input data.

    Notes:
    ------
    This method takes a pandas DataFrame as input data and converts it into a
    torch tensor. It specifically selects columns corresponding to the model's
    input names (variables), and the resulting tensor is configured with the data
    type and device settings from the generator.
    """
    return torch.tensor(data[self.model_input_names].to_numpy(), **self.tkwargs)

xopt.generators.bayesian.bayesian_generator.BayesianGenerator.get_optimum

get_optimum()

select the best point(s) given by the model using the Posterior mean

Source code in xopt/generators/bayesian/bayesian_generator.py

def get_optimum(self):
    """select the best point(s) given by the
    model using the Posterior mean"""
    c_posterior_mean = ConstrainedMCAcquisitionFunction(
        self.model,
        qUpperConfidenceBound(
            model=self.model, beta=0.0, objective=self._get_objective()
        ),
        self._get_constraint_callables(),
    )

    result = self.numerical_optimizer.optimize(
        c_posterior_mean, self._get_bounds(), 1
    )

    return self._process_candidates(result)

xopt.generators.bayesian.bayesian_generator.BayesianGenerator.get_training_data

get_training_data(data)

Get training data used to train the GP model

If a turbo controller is specified with the flag restrict_model_data this will return a subset of data that is inside the trust region.

Parameters:

data : pd.DataFrame The data in the form of a pandas DataFrame.

Returns:

data : pd.DataFrame A subset of data used to train the model form of a pandas DataFrame.

Source code in xopt/generators/bayesian/bayesian_generator.py

def get_training_data(self, data: pd.DataFrame) -> pd.DataFrame:
    """
    Get training data used to train the GP model

    If a turbo controller is specified with the flag `restrict_model_data` this
    will return a subset of data that is inside the trust region.

    Parameters:
    -----------
    data : pd.DataFrame
        The data in the form of a pandas DataFrame.

    Returns:
    --------
    data : pd.DataFrame
        A subset of data used to train the model form of a pandas DataFrame.

    """
    if self.turbo_controller is not None:
        if self.turbo_controller.restrict_model_data:
            data = self.turbo_controller.get_data_in_trust_region(data, self)

    return data

xopt.generators.bayesian.bayesian_generator.BayesianGenerator.propose_candidates

propose_candidates(model, n_candidates=1)

given a GP model, propose candidates by numerically optimizing the acquisition function

Source code in xopt/generators/bayesian/bayesian_generator.py

def propose_candidates(self, model, n_candidates=1):
    """
    given a GP model, propose candidates by numerically optimizing the
    acquisition function

    """
    # update TurBO state if used with the last `n_candidates` points
    if self.turbo_controller is not None:
        self.turbo_controller.update_state(self, n_candidates)

    # calculate optimization bounds
    bounds = self._get_optimization_bounds()

    # get acquisition function
    acq_funct = self.get_acquisition(model)

    # get initial candidates to start acquisition function optimization
    initial_points = self._get_initial_conditions(n_candidates)

    # get candidates -- grid optimizer does not support batch_initial_conditions
    if isinstance(self.numerical_optimizer, GridOptimizer):
        candidates = self.numerical_optimizer.optimize(
            acq_funct, bounds, n_candidates
        )
    else:
        candidates = self.numerical_optimizer.optimize(
            acq_funct, bounds, n_candidates, batch_initial_conditions=initial_points
        )
    return candidates

xopt.generators.bayesian.bayesian_generator.BayesianGenerator.train_model

train_model(data=None, update_internal=True)

Returns a ModelListGP containing independent models for the objectives and constraints

Source code in xopt/generators/bayesian/bayesian_generator.py

def train_model(self, data: pd.DataFrame = None, update_internal=True) -> Module:
    """
    Returns a ModelListGP containing independent models for the objectives and
    constraints

    """
    if data is None:
        data = self.get_training_data(self.data)
    if data.empty:
        raise ValueError("no data available to build model")

    # get input bounds
    variable_bounds = deepcopy(self.vocs.variables)

    # if turbo restrict points is true then set the bounds to the trust region
    # bounds
    if self.turbo_controller is not None:
        if self.turbo_controller.restrict_model_data:
            variable_bounds = dict(
                zip(
                    self.vocs.variable_names,
                    self.turbo_controller.get_trust_region(self).numpy().T,
                )
            )

    # add fixed feature bounds if requested
    if self.fixed_features is not None:
        # get bounds for each fixed_feature (vocs bounds take precedent)
        for key in self.fixed_features:
            if key not in variable_bounds:
                if key not in data:
                    raise KeyError(
                        "generator data needs to contain fixed feature "
                        f"column name `{key}`"
                    )
                f_data = data[key]
                bounds = [f_data.min(), f_data.max()]
                if bounds[1] - bounds[0] < 1e-8:
                    bounds[1] = bounds[0] + 1e-8
                variable_bounds[key] = bounds

    _model = self.gp_constructor.build_model(
        self.model_input_names,
        self.vocs.output_names,
        data,
        {name: variable_bounds[name] for name in self.model_input_names},
        **self.tkwargs,
    )

    if update_internal:
        self.model = _model
    return _model

xopt.generators.bayesian.bayesian_generator.BayesianGenerator.validate_turbo_controller

validate_turbo_controller(value, info)

note default behavior is no use of turbo

Source code in xopt/generators/bayesian/bayesian_generator.py

@field_validator("turbo_controller", mode="before")
def validate_turbo_controller(cls, value, info: ValidationInfo):
    """note default behavior is no use of turbo"""
    controller_dict = {
        "optimize": OptimizeTurboController,
        "safety": SafetyTurboController,
    }
    value = validate_turbo_controller_base(value, controller_dict, info)

    return value

xopt.generators.bayesian.bayesian_generator.BayesianGenerator.visualize_model

visualize_model(**kwargs)

displays the GP models

Source code in xopt/generators/bayesian/bayesian_generator.py

def visualize_model(self, **kwargs):
    """displays the GP models"""
    return visualize_generator_model(self, **kwargs)

xopt.generators.bayesian.bayesian_exploration.BayesianExplorationGenerator

BayesianExplorationGenerator(**kwargs)

Bases: BayesianGenerator

Source code in xopt/generator.py

def __init__(self, **kwargs):
    """
    Initialize the generator.

    """
    super().__init__(**kwargs)
    logger.info(f"Initialized generator {self.name}")

xopt.generators.bayesian.mobo.MOBOGenerator

MOBOGenerator(**kwargs)

Bases: MultiObjectiveBayesianGenerator

Source code in xopt/generator.py

def __init__(self, **kwargs):
    """
    Initialize the generator.

    """
    super().__init__(**kwargs)
    logger.info(f"Initialized generator {self.name}")

Functions

xopt.generators.bayesian.mobo.MOBOGenerator.get_acquisition

get_acquisition(model)

Returns a function that can be used to evaluate the acquisition function

Source code in xopt/generators/bayesian/mobo.py

def get_acquisition(self, model):
    """
    Returns a function that can be used to evaluate the acquisition function
    """
    if model is None:
        raise ValueError("model cannot be None")

    # get base acquisition function
    acq = self._get_acquisition(model)

    # apply fixed features if specified in the generator
    if self.fixed_features is not None:
        # get input dim
        dim = len(self.model_input_names)
        columns = []
        values = []
        for name, value in self.fixed_features.items():
            columns += [self.model_input_names.index(name)]
            values += [value]

        acq = FixedFeatureAcquisitionFunction(
            acq_function=acq, d=dim, columns=columns, values=values
        )

    return acq

xopt.generators.bayesian.upper_confidence_bound.UpperConfidenceBoundGenerator

UpperConfidenceBoundGenerator(**kwargs)

Bases: BayesianGenerator

Source code in xopt/generator.py

def __init__(self, **kwargs):
    """
    Initialize the generator.

    """
    super().__init__(**kwargs)
    logger.info(f"Initialized generator {self.name}")