{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Basics of Trust Region Controllers in Xopt" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Trust Region Bayesian Optimization (TuRBO) is an advanced optimization algorithm designed for solving high-dimensional black-box optimization problems. It combines the strengths of Bayesian Optimization (BO) with trust region methods to improve scalability and efficiency.\n", "\n", "### Key Features:\n", "1. **Trust Regions**:\n", " - TuRBO uses local trust regions to focus the search in promising areas of the parameter space.\n", " - Each trust region is a bounded subspace where the optimization is performed, and its size is dynamically adjusted based on the success of the optimization.\n", "\n", "2. **Bayesian Surrogate Model**:\n", " - A Gaussian Process (GP) or other surrogate models are used to approximate the objective function.\n", " - This surrogate model is used to predict the objective function and guide the search as well as define the size of the trust region.\n", "\n", "4. **Adaptivity**:\n", " - The algorithm adapts the size of the trust region based on the success or failure of the optimization steps. If the optimization within a trust region is successful, the region expands; otherwise, it shrinks.\n", "\n", "### Advantages:\n", "- Scales better to high-dimensional problems compared to standard Bayesian Optimization.\n", "- Efficiently balances exploration and exploitation within trust regions.\n", "\n", "### Disadvantages:\n", "- Severely restricts exploration of the parameter space potentially leading to convergence to local minima, thus making it sensitive to initial sampling points.\n", "- Introduces additional algorithm hyperparameters which can cause issues. \n", "- May struggle with noisy objective functions or discontinuous landscapes. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Defining a TuRBO Controller\n", "Currently, Xopt supports 3 different TuRBO controller types, the most basic of which is the `OptimizeTurboController`. To create this controller we need to define our optimization problem and some data." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "execution": { "iopub.execute_input": "2025-05-30T15:50:25.743819Z", "iopub.status.busy": "2025-05-30T15:50:25.743657Z", "iopub.status.idle": "2025-05-30T15:50:27.140101Z", "shell.execute_reply": "2025-05-30T15:50:27.139587Z" } }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
fxy
020.847724-4.320888-1.475685
123.5819584.8244950.553360
29.462963-2.4295241.886896
311.3453913.265719-0.824906
420.360560-1.907623-4.089197
58.1313170.0928062.850036
612.9111513.5488500.562862
73.6404781.2237791.463845
825.0583112.4277044.377735
921.2201713.125956-3.383574
\n", "
" ], "text/plain": [ " f x y\n", "0 20.847724 -4.320888 -1.475685\n", "1 23.581958 4.824495 0.553360\n", "2 9.462963 -2.429524 1.886896\n", "3 11.345391 3.265719 -0.824906\n", "4 20.360560 -1.907623 -4.089197\n", "5 8.131317 0.092806 2.850036\n", "6 12.911151 3.548850 0.562862\n", "7 3.640478 1.223779 1.463845\n", "8 25.058311 2.427704 4.377735\n", "9 21.220171 3.125956 -3.383574" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import numpy as np\n", "from xopt import VOCS\n", "import pandas as pd\n", "\n", "\n", "# create evaluation function\n", "def sphere_function(inputs):\n", " \"\"\"\n", " 2D Sphere objective function.\n", " Compatible with Xopt.\n", " \"\"\"\n", " x, y = inputs[\"x\"], inputs[\"y\"]\n", " return {\"f\": np.sum(np.square(np.stack([x, y], axis=-1)), axis=-1)}\n", "\n", "\n", "# create a VOCS object\n", "vocs = VOCS(\n", " variables={\"x\": {-5, 5}, \"y\": {-5, 5}},\n", " objectives={\"f\": \"MINIMIZE\"},\n", ")\n", "\n", "# random sample 10 points\n", "x0 = vocs.random_inputs(10)\n", "\n", "# evaluate the function at the random points\n", "f = []\n", "for i in range(len(x0)):\n", " f += [sphere_function(x0[i]) | x0[i]]\n", "\n", "# print the results\n", "data = pd.DataFrame(f)\n", "data" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Create the ExpectedImprovementGenerator and train the GP model\n", "Here we create the ExpectedImprovementGenerator, add data to the generator, and train the model from the data." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "execution": { "iopub.execute_input": "2025-05-30T15:50:27.168078Z", "iopub.status.busy": "2025-05-30T15:50:27.167587Z", "iopub.status.idle": "2025-05-30T15:50:28.509711Z", "shell.execute_reply": "2025-05-30T15:50:28.509222Z" } }, "outputs": [ { "data": { "text/plain": [ "ModelListGP(\n", " (models): ModuleList(\n", " (0): SingleTaskGP(\n", " (likelihood): GaussianLikelihood(\n", " (noise_covar): HomoskedasticNoise(\n", " (noise_prior): GammaPrior()\n", " (raw_noise_constraint): GreaterThan(1.000E-04)\n", " )\n", " )\n", " (mean_module): ConstantMean()\n", " (covar_module): RBFKernel(\n", " (lengthscale_prior): LogNormalPrior()\n", " (raw_lengthscale_constraint): GreaterThan(2.500E-02)\n", " )\n", " (outcome_transform): Standardize()\n", " (input_transform): Normalize()\n", " )\n", " )\n", " (likelihood): LikelihoodList(\n", " (likelihoods): ModuleList(\n", " (0): GaussianLikelihood(\n", " (noise_covar): HomoskedasticNoise(\n", " (noise_prior): GammaPrior()\n", " (raw_noise_constraint): GreaterThan(1.000E-04)\n", " )\n", " )\n", " )\n", " )\n", ")" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from xopt.generators.bayesian import ExpectedImprovementGenerator\n", "\n", "generator = ExpectedImprovementGenerator(vocs=vocs) # create the generator\n", "generator.gp_constructor.use_low_noise_prior = True\n", "generator.add_data(data) # add the data to the generator\n", "generator.train_model() # train the model" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Create the Optimize Turbo Controller\n", "Here we create the controller and view the different parameters with their descriptions." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "execution": { "iopub.execute_input": "2025-05-30T15:50:28.511533Z", "iopub.status.busy": "2025-05-30T15:50:28.511259Z", "iopub.status.idle": "2025-05-30T15:50:28.516055Z", "shell.execute_reply": "2025-05-30T15:50:28.515550Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "Turbo controller for optimization tasks.\n", "\n", "Attributes:\n", "-----------\n", "name : str\n", " The name of the controller.\n", "best_value : Optional[float]\n", " The best value found so far.\n", "\n", "Methods:\n", "--------\n", "vocs_validation(cls, info)\n", " Validate the VOCS for the controller.\n", "minimize(self) -> bool\n", " Check if the objective is to minimize.\n", "_set_best_point_value(self, data)\n", " Set the best point value based on the data.\n", "update_state(self, generator, previous_batch_size: int = 1) -> None\n", " Update the state of the controller.\n", "\n", "--------------------\n", "Field: vocs\n", " Description: VOCS object\n", " Type: \n", " Default: PydanticUndefined\n", " Value: variables={'x': [-5.0, 5.0], 'y': [-5.0, 5.0]} constraints={} objectives={'f': 'MINIMIZE'} constants={} observables=[]\n", "--------------------\n", "Field: dim\n", " Description: number of dimensions in the optimization problem\n", " Type: \n", " Default: None\n", " Value: 2\n", "--------------------\n", "Field: batch_size\n", " Description: number of trust regions to use\n", " Type: \n", " Default: 1\n", " Value: 1\n", "--------------------\n", "Field: length\n", " Description: base length of trust region\n", " Type: \n", " Default: 0.25\n", " Value: 0.25\n", "--------------------\n", "Field: length_min\n", " Description: minimum base length of trust region\n", " Type: \n", " Default: 0.0078125\n", " Value: 0.0078125\n", "--------------------\n", "Field: length_max\n", " Description: maximum base length of trust region\n", " Type: \n", " Default: 2.0\n", " Value: 2.0\n", "--------------------\n", "Field: failure_counter\n", " Description: number of failures since reset\n", " Type: \n", " Default: 0\n", " Value: 0\n", "--------------------\n", "Field: failure_tolerance\n", " Description: number of failures to trigger a trust region expansion\n", " Type: \n", " Default: None\n", " Value: 2\n", "--------------------\n", "Field: success_counter\n", " Description: number of successes since reset\n", " Type: \n", " Default: 0\n", " Value: 0\n", "--------------------\n", "Field: success_tolerance\n", " Description: number of successes to trigger a trust region contraction\n", " Type: \n", " Default: None\n", " Value: 2\n", "--------------------\n", "Field: center_x\n", " Description: center point of trust region\n", " Type: typing.Optional[typing.Dict[str, float]]\n", " Default: None\n", " Value: None\n", "--------------------\n", "Field: scale_factor\n", " Description: multiplier to increase or decrease trust region\n", " Type: \n", " Default: 2.0\n", " Value: 2.0\n", "--------------------\n", "Field: restrict_model_data\n", " Description: flag to restrict model data to within the trust region\n", " Type: typing.Optional[bool]\n", " Default: True\n", " Value: True\n", "--------------------\n", "Field: name\n", " Description: name of the Turbo controller\n", " Type: \n", " Default: OptimizeTurboController\n", " Value: OptimizeTurboController\n", "--------------------\n", "Field: best_value\n", " Description: best objective value found so far\n", " Type: typing.Optional[float]\n", " Default: None\n", " Value: None\n", "--------------------\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "/tmp/ipykernel_4717/1076008262.py:9: PydanticDeprecatedSince211: Accessing the 'model_fields' attribute on the instance is deprecated. Instead, you should access this attribute from the model class. Deprecated in Pydantic V2.11 to be removed in V3.0.\n", " for field_name, field in turbo_controller.model_fields.items():\n" ] } ], "source": [ "from xopt.generators.bayesian.turbo import OptimizeTurboController\n", "\n", "turbo_controller = OptimizeTurboController(vocs=vocs)\n", "\n", "print(turbo_controller.__doc__)\n", "print(\"-\" * 20)\n", "\n", "# examine the attributes of the controller\n", "for field_name, field in turbo_controller.model_fields.items():\n", " print(f\"Field: {field_name}\")\n", " print(f\" Description: {field.description}\")\n", " print(f\" Type: {field.annotation}\")\n", " print(f\" Default: {field.default}\")\n", " print(f\" Value: {getattr(turbo_controller, field_name)}\")\n", " print(\"-\" * 20)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Getting the Trust Region\n", "Here we get the current trust region \n" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "execution": { "iopub.execute_input": "2025-05-30T15:50:28.517665Z", "iopub.status.busy": "2025-05-30T15:50:28.517498Z", "iopub.status.idle": "2025-05-30T15:50:28.521592Z", "shell.execute_reply": "2025-05-30T15:50:28.521150Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Trust Region: tensor([[-5., -5.],\n", " [ 5., 5.]], dtype=torch.float64)\n" ] } ], "source": [ "trust_region = turbo_controller.get_trust_region(\n", " generator=generator\n", ") # get the trust region of the model\n", "print(f\"Trust Region: {trust_region}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Update the trust region\n", "Add another data point to the generator (as if we performed one optimization step) and update the turbo controller. We will add a point that improves over the best function value measured so far so this measurement will count as a success." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "execution": { "iopub.execute_input": "2025-05-30T15:50:28.523249Z", "iopub.status.busy": "2025-05-30T15:50:28.522887Z", "iopub.status.idle": "2025-05-30T15:50:28.526649Z", "shell.execute_reply": "2025-05-30T15:50:28.526219Z" } }, "outputs": [], "source": [ "# add a new point to the generator\n", "new_point = pd.DataFrame({\"x\": [0.0], \"y\": [0.0], \"f\": [0.0]})\n", "generator.add_data(new_point) # add the new point to the generator" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "execution": { "iopub.execute_input": "2025-05-30T15:50:28.528351Z", "iopub.status.busy": "2025-05-30T15:50:28.528012Z", "iopub.status.idle": "2025-05-30T15:50:28.632303Z", "shell.execute_reply": "2025-05-30T15:50:28.631841Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "New Trust Region: tensor([[-5.0000, -2.8340],\n", " [-3.1706, -0.1173]], dtype=torch.float64)\n", "Number of successes: 1\n", "Number of failures: 0\n", "Base length scale: 0.25\n" ] } ], "source": [ "generator.train_model() # train the model again\n", "\n", "# update the TuRBO controller\n", "turbo_controller.update_state(generator)\n", "\n", "# get the new trust region\n", "trust_region = turbo_controller.get_trust_region(\n", " generator=generator\n", ") # get the trust region of the model\n", "print(f\"New Trust Region: {trust_region}\")\n", "\n", "# get the number of successes and failures\n", "print(f\"Number of successes: {turbo_controller.success_counter}\")\n", "print(f\"Number of failures: {turbo_controller.failure_counter}\")\n", "\n", "# get the base length scale of the trust region\n", "print(f\"Base length scale: {turbo_controller.length}\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.13.3" } }, "nbformat": 4, "nbformat_minor": 2 }