Training and Tuning AGNBoost Models from Scratch¶

This notebook demonstrates how to train and tune your own AGNBoost models. We'll walk through:

1. Displaying what trained models are available
2. Training new models
3. Tuning the models

Let's start by importing the necessary libraries and loading our data.

In [2]:

# Set agnboost folder as root
import os

# Navigate to the repository root (parent directory of notebooks/)
os.chdir('..')

# Import necessary libraries
import numpy as np
import pandas as pd
from agnboost import dataset, model
import seaborn as sns
import matplotlib.pyplot as plt

# Set random seed for reproducibility
np.random.seed(123)

print("AGNBoost Basic Usage Tutorial")
print("=" * 40)

# Set agnboost folder as root import os # Navigate to the repository root (parent directory of notebooks/) os.chdir('..') # Import necessary libraries import numpy as np import pandas as pd from agnboost import dataset, model import seaborn as sns import matplotlib.pyplot as plt # Set random seed for reproducibility np.random.seed(123) print("AGNBoost Basic Usage Tutorial") print("=" * 40)

2025-05-31 21:17:14.384 | INFO     | agnboost.config:<module>:11 - PROJ_ROOT path is: /home/kurt/Documents/agnboost

AGNBoost Basic Usage Tutorial
========================================

Loading the Data + Train/Val/Test Split¶

We'll use the Catalog class to load our astronomical dataset. The cigale_mock_small.csv file contains is a small set of mock NIRCam+MIRI CIGALE galaxies for demonstration purposes.

Let's load the data and then split the data so we have 60% for training, 20% for validation, and 20% for testing.

In [23]:

# Load the astronomical data using the Catalog class with all NIRCam+MIRI bands
catalog = dataset.Catalog(path="data/cigale_mock_small.csv",summarize = False)

# Create train/validation/test splitsget_train_val_test_sizes
catalog.split_data(test_size=0.2, val_size=0.2, random_state=42)

# Get split information
split_info = catalog.get_train_val_test_sizes()
print("Data split summary:")
print(f"  Total samples: {split_info['total']}")
print(f"  Training: {split_info['train']['size']} ({split_info['train']['percentage']:.1f}%)")
print(f"  Validation: {split_info['validation']['size']} ({split_info['validation']['percentage']:.1f}%)")
print(f"  Test: {split_info['test']['size']} ({split_info['test']['percentage']:.1f}%)")

# Load the astronomical data using the Catalog class with all NIRCam+MIRI bands catalog = dataset.Catalog(path="data/cigale_mock_small.csv",summarize = False) # Create train/validation/test splitsget_train_val_test_sizes catalog.split_data(test_size=0.2, val_size=0.2, random_state=42) # Get split information split_info = catalog.get_train_val_test_sizes() print("Data split summary:") print(f" Total samples: {split_info['total']}") print(f" Training: {split_info['train']['size']} ({split_info['train']['percentage']:.1f}%)") print(f" Validation: {split_info['validation']['size']} ({split_info['validation']['percentage']:.1f}%)") print(f" Test: {split_info['test']['size']} ({split_info['test']['percentage']:.1f}%)")

Current working directory: /home/kurt/Documents/agnboost
Looking for bands file at: /home/kurt/Documents/agnboost/agnboost/allowed_bands.json
[INFO] Loaded bands file metadata: This file contains the allowed photometric bands for JWST
[INFO] Loaded 11 allowed bands from agnboost/allowed_bands.json
[INFO] Attempting to load file with delimiter: ','
[INFO] Successfully loaded data with 1000 rows.
[INFO] Found 11 valid band columns:
[INFO]   - jwst.nircam.F115W (F115W): 1.154 μm
[INFO]   - jwst.nircam.F150W (F150W): 1.501 μm
[INFO]   - jwst.nircam.F200W (F200W): 1.988 μm
[INFO]   - jwst.nircam.F277W (F277W): 2.776 μm
[INFO]   - jwst.nircam.F356W (F356W): 3.565 μm
[INFO]   - jwst.nircam.F410M (F410M): 4.083 μm
[INFO]   - jwst.nircam.F444W (F444W): 4.402 μm
[INFO]   - jwst.miri.F770W (F770W): 7.7 μm
[INFO]   - jwst.miri.F1000W (F1000W): 10.0 μm
[INFO]   - jwst.miri.F1500W (F1500W): 15.0 μm
[INFO]   - jwst.miri.F2100W (F2100W): 21.0 μm
Data split summary:
  Total samples: 1000
  Training: 600 (60.0%)
  Validation: 200 (20.0%)
  Test: 200 (20.0%)

Creating Features¶

We then need to create our features. We will use the default features (phots + colors + colors^2)/

In [4]:

# Create features for modeling
catalog.create_feature_dataframe()

# Get information about created features
features = catalog.get_features()
print(f"Feature engineering complete:")
print(f"  Feature dataframe shape: {features.shape}")

# Create features for modeling catalog.create_feature_dataframe() # Get information about created features features = catalog.get_features() print(f"Feature engineering complete:") print(f" Feature dataframe shape: {features.shape}")

[INFO] Created feature dataframe with 121 columns and 1000 rows.
[INFO] Created features are: ['jwst.nircam.F115W', 'jwst.nircam.F150W', 'jwst.nircam.F200W', 'jwst.nircam.F277W', 'jwst.nircam.F356W', 'jwst.nircam.F410M', 'jwst.nircam.F444W', 'jwst.miri.F770W', 'jwst.miri.F1000W', 'jwst.miri.F1500W', 'jwst.miri.F2100W', 'F2100W/F1500W', 'F2100W/F1000W', 'F2100W/F770W', 'F2100W/F444W', 'F2100W/F410M', 'F2100W/F356W', 'F2100W/F277W', 'F2100W/F200W', 'F2100W/F150W', 'F2100W/F115W', 'F1500W/F1000W', 'F1500W/F770W', 'F1500W/F444W', 'F1500W/F410M', 'F1500W/F356W', 'F1500W/F277W', 'F1500W/F200W', 'F1500W/F150W', 'F1500W/F115W', 'F1000W/F770W', 'F1000W/F444W', 'F1000W/F410M', 'F1000W/F356W', 'F1000W/F277W', 'F1000W/F200W', 'F1000W/F150W', 'F1000W/F115W', 'F770W/F444W', 'F770W/F410M', 'F770W/F356W', 'F770W/F277W', 'F770W/F200W', 'F770W/F150W', 'F770W/F115W', 'F444W/F410M', 'F444W/F356W', 'F444W/F277W', 'F444W/F200W', 'F444W/F150W', 'F444W/F115W', 'F410M/F356W', 'F410M/F277W', 'F410M/F200W', 'F410M/F150W', 'F410M/F115W', 'F356W/F277W', 'F356W/F200W', 'F356W/F150W', 'F356W/F115W', 'F277W/F200W', 'F277W/F150W', 'F277W/F115W', 'F200W/F150W', 'F200W/F115W', 'F150W/F115W', 'F2100W/F1500W^2', 'F2100W/F1000W^2', 'F2100W/F770W^2', 'F2100W/F444W^2', 'F2100W/F410M^2', 'F2100W/F356W^2', 'F2100W/F277W^2', 'F2100W/F200W^2', 'F2100W/F150W^2', 'F2100W/F115W^2', 'F1500W/F1000W^2', 'F1500W/F770W^2', 'F1500W/F444W^2', 'F1500W/F410M^2', 'F1500W/F356W^2', 'F1500W/F277W^2', 'F1500W/F200W^2', 'F1500W/F150W^2', 'F1500W/F115W^2', 'F1000W/F770W^2', 'F1000W/F444W^2', 'F1000W/F410M^2', 'F1000W/F356W^2', 'F1000W/F277W^2', 'F1000W/F200W^2', 'F1000W/F150W^2', 'F1000W/F115W^2', 'F770W/F444W^2', 'F770W/F410M^2', 'F770W/F356W^2', 'F770W/F277W^2', 'F770W/F200W^2', 'F770W/F150W^2', 'F770W/F115W^2', 'F444W/F410M^2', 'F444W/F356W^2', 'F444W/F277W^2', 'F444W/F200W^2', 'F444W/F150W^2', 'F444W/F115W^2', 'F410M/F356W^2', 'F410M/F277W^2', 'F410M/F200W^2', 'F410M/F150W^2', 'F410M/F115W^2', 'F356W/F277W^2', 'F356W/F200W^2', 'F356W/F150W^2', 'F356W/F115W^2', 'F277W/F200W^2', 'F277W/F150W^2', 'F277W/F115W^2', 'F200W/F150W^2', 'F200W/F115W^2', 'F150W/F115W^2']
Feature engineering complete:
  Feature dataframe shape: (1000, 121)

Creating a new model¶

First, let's take a look at the columns of our saved data to see what our regression options are.

In [5]:

# Print columns in data
print( catalog.get_data().columns )

# Print columns in data print( catalog.get_data().columns )

Index(['IRAC1', 'IRAC2', 'IRAC3', 'IRAC4', 'hst.acs.wfc.F606W',
       'hst.acs.wfc.F814W', 'hst.wfc3.ir.F125W', 'hst.wfc3.ir.F140W',
       'hst.wfc3.ir.F160W', 'jwst.miri.F1000W', 'jwst.miri.F1280W',
       'jwst.miri.F1500W', 'jwst.miri.F1800W', 'jwst.miri.F2100W',
       'jwst.miri.F770W', 'jwst.nircam.F115W', 'jwst.nircam.F150W',
       'jwst.nircam.F200W', 'jwst.nircam.F277W', 'jwst.nircam.F356W',
       'jwst.nircam.F410M', 'jwst.nircam.F444W', 'sfh.sfr100Myrs',
       'stellar.m_star', 'agn.fracAGN', 'universe.redshift'],
      dtype='object')

Let's say we wanted to create our own model to estimate the agn.fracAGN (i.e. frac$_\text{AGN}$), parameter (recall that this is the fraction of 3-30 micron light attributable to an AGN power law). We first need to instantiate an AGNBoost model with agn.fracAGN as the target variable. We already know that frac$_\text{AGN}$ is bounded [0,1), but let's first quickly visualize the distribution of agn.fracAGN to decide what type of probability distribution to model.

In [10]:

fig, ax = plt.subplots()

bins_width = 0.25
bins = np.arange(0, 1 + bins_width, bins_width)
fagn_data = catalog.get_data()['agn.fracAGN']
sns.histplot( x = catalog.get_data()['agn.fracAGN'], ax = ax, bins = bins, color = 'xkcd:cobalt blue',  element = 'step', fill = True, alpha = 0.3)

print(f"min fracAGN: {np.min(fagn_data )}")
print(f"max fracAGN: {np.max(fagn_data )}")

fig, ax = plt.subplots() bins_width = 0.25 bins = np.arange(0, 1 + bins_width, bins_width) fagn_data = catalog.get_data()['agn.fracAGN'] sns.histplot( x = catalog.get_data()['agn.fracAGN'], ax = ax, bins = bins, color = 'xkcd:cobalt blue', element = 'step', fill = True, alpha = 0.3) print(f"min fracAGN: {np.min(fagn_data )}") print(f"max fracAGN: {np.max(fagn_data )}")

min fracAGN: 0.0
max fracAGN: 0.99

As expected, the agn.fracAGN variable is bounded [0, 1), so we will model it with a zero-inflated beta distribution (ZABeta). Let us instantiate a AGNBoost model with agn.fracAGN as the target variable, and using ZABeta as the modeled probabiltiy distribution:

In [12]:

# Initialize an AGNBoost model. The target variable is the name of the target variable column, and its value in the passed dictionary is the distribution used to model it.
agnboost_m = model.AGNBoost( feature_names = catalog.get_feature_names(),
                          target_variables = {'agn.fracAGN' : 'ZABeta'},
                         )

# Initialize an AGNBoost model. The target variable is the name of the target variable column, and its value in the passed dictionary is the distribution used to model it. agnboost_m = model.AGNBoost( feature_names = catalog.get_feature_names(), target_variables = {'agn.fracAGN' : 'ZABeta'}, )

In [ ]:

Model Tuning¶

We have created the AGNBoost model, but before we can train a production model to predict agn.fracAGN, we need to tune the hyperparameters of AGNBoost.

The XGBoostLSS models themselves generally have 3 tunable hyperparamters:

1. Stabilization: "None", "MAD" (i.e., L1), "L2"
2. response_sn: "exp", "softplus"
3. loss_fn: "nll", "crps"

For the details of these hyperparameters, and the options for each individual XGBoostLSS distribution, please refer to the XGBoostLSS documentation, and the XGBoostLSS code on GitHub. For a full list of XGBoost booster hyperparameters, refer to the XGBoost documentation.

To perform hyperparameter tuning, we need to create a python dictionary that defines what hyperparameters to vary, and the possible values or value ranges of the parameter. This needs to be formatted a specific way:

- Float/Int sample_type
    - {"param_name": ["sample_type", low, high, log]}
        - sample_type: str, Type of sampling, e.g., "float" or "int"
        - low: int, Lower endpoint of the range of suggested values
        - high: int, Upper endpoint of the range of suggested values
        - log: bool, Flag to sample the value from the log domain or not
    - Example: {"eta": "float", low=1e-5, high=1, log=True]}

- Categorical (or None) sample_type
    - {"param_name": ["sample_type", ["choice1", "choice2", "choice3", "..."]]}
        - sample_type: str, Type of sampling, either "categorical"
        - choice1, choice2, choice3, ...: str, Possible choices for the parameter
    - Example: {"booster": ["categorical", ["gbtree", "dart"]]}

Note that the "categorical" and "None" data types are equivalent and used interchangeably. Let's pick a few XGBoostLSS and XGBoost hyperparameters to tune:

In [24]:

param_dict = {
    "stabilization":          ["categorical", ["None", "MAD", "L2"]],
    "response_fn":          ["categorical", ["softplus", "exp"]],

    "eta":              ["none", [.5]],
    "tree_method":    ["categorical", ["hist","approx"] ],
    "max_depth":        ["int",   {"low": 3,      "high": 10,    "log": False}],
    "gamma":            ["float", {"low": 1e-8,   "high": 40,    "log": True}], 
    "min_child_weight": ["int", {"low": 1,   "high": 250,  "log": False}],
    "lambda":            ["float", {"low": 1,   "high": 150,    "log": True}], 

    "device" : ["categorical", ["cpu"] ]
}

param_dict = { "stabilization": ["categorical", ["None", "MAD", "L2"]], "response_fn": ["categorical", ["softplus", "exp"]], "eta": ["none", [.5]], "tree_method": ["categorical", ["hist","approx"] ], "max_depth": ["int", {"low": 3, "high": 10, "log": False}], "gamma": ["float", {"low": 1e-8, "high": 40, "log": True}], "min_child_weight": ["int", {"low": 1, "high": 250, "log": False}], "lambda": ["float", {"low": 1, "high": 150, "log": True}], "device" : ["categorical", ["cpu"] ] }

Sidenote: GPU usage¶

If you have a NVIDIA CUDA compatible GPU, you can certainly make use of it to train AGNBoost models. To do this, ensure that your XGBoost instllation is gpu-compatible, see the example here, and to install gpu-compatible xgboost, see the docs here. To use the gpu for training, you need to set "device" : ["categorical", ["gpu"] ]. Just note that it is really only worth using the GPU if you are using about a million or so points of data for training. Any less than that and using the CPU will generally be faster.

Now that we have our parameter dictionary, we can begin the tuning process. Fortuantely, this is very simple:

In [25]:

best_params_1stpass = agnboost_m.tune_model( model_name = 'agn.fracAGN',
                                             param_grid = param_dict,
                                             dtune = catalog,
                                            max_minutes = 10, #We will tune for 10 minutes. You will probably want to tune for much longer!
                                             split_type = 'trainval'  #Options: train, val, test, trainval. trainval uses the combined train+val test split
                                           )

best_params_1stpass = agnboost_m.tune_model( model_name = 'agn.fracAGN', param_grid = param_dict, dtune = catalog, max_minutes = 10, #We will tune for 10 minutes. You will probably want to tune for much longer! split_type = 'trainval' #Options: train, val, test, trainval. trainval uses the combined train+val test split )

2025-05-31 21:59:39,310 - AGNBoost.AGNBoost - WARNING - Catalog object passsed. Taking the features and labels of the trainval set stored in the passed Catalog.
``multivariate`` option is an experimental feature. The interface can change in the future.
[I 2025-05-31 21:59:39,337] A new study created in memory with name: no-name-15235329-2d3f-4858-ab06-f5bb2604f0d8

[INFO] Created feature dataframe with 121 columns and 1000 rows.
[INFO] Created features are: ['jwst.nircam.F115W', 'jwst.nircam.F150W', 'jwst.nircam.F200W', 'jwst.nircam.F277W', 'jwst.nircam.F356W', 'jwst.nircam.F410M', 'jwst.nircam.F444W', 'jwst.miri.F770W', 'jwst.miri.F1000W', 'jwst.miri.F1500W', 'jwst.miri.F2100W', 'F2100W/F1500W', 'F2100W/F1000W', 'F2100W/F770W', 'F2100W/F444W', 'F2100W/F410M', 'F2100W/F356W', 'F2100W/F277W', 'F2100W/F200W', 'F2100W/F150W', 'F2100W/F115W', 'F1500W/F1000W', 'F1500W/F770W', 'F1500W/F444W', 'F1500W/F410M', 'F1500W/F356W', 'F1500W/F277W', 'F1500W/F200W', 'F1500W/F150W', 'F1500W/F115W', 'F1000W/F770W', 'F1000W/F444W', 'F1000W/F410M', 'F1000W/F356W', 'F1000W/F277W', 'F1000W/F200W', 'F1000W/F150W', 'F1000W/F115W', 'F770W/F444W', 'F770W/F410M', 'F770W/F356W', 'F770W/F277W', 'F770W/F200W', 'F770W/F150W', 'F770W/F115W', 'F444W/F410M', 'F444W/F356W', 'F444W/F277W', 'F444W/F200W', 'F444W/F150W', 'F444W/F115W', 'F410M/F356W', 'F410M/F277W', 'F410M/F200W', 'F410M/F150W', 'F410M/F115W', 'F356W/F277W', 'F356W/F200W', 'F356W/F150W', 'F356W/F115W', 'F277W/F200W', 'F277W/F150W', 'F277W/F115W', 'F200W/F150W', 'F200W/F115W', 'F150W/F115W', 'F2100W/F1500W^2', 'F2100W/F1000W^2', 'F2100W/F770W^2', 'F2100W/F444W^2', 'F2100W/F410M^2', 'F2100W/F356W^2', 'F2100W/F277W^2', 'F2100W/F200W^2', 'F2100W/F150W^2', 'F2100W/F115W^2', 'F1500W/F1000W^2', 'F1500W/F770W^2', 'F1500W/F444W^2', 'F1500W/F410M^2', 'F1500W/F356W^2', 'F1500W/F277W^2', 'F1500W/F200W^2', 'F1500W/F150W^2', 'F1500W/F115W^2', 'F1000W/F770W^2', 'F1000W/F444W^2', 'F1000W/F410M^2', 'F1000W/F356W^2', 'F1000W/F277W^2', 'F1000W/F200W^2', 'F1000W/F150W^2', 'F1000W/F115W^2', 'F770W/F444W^2', 'F770W/F410M^2', 'F770W/F356W^2', 'F770W/F277W^2', 'F770W/F200W^2', 'F770W/F150W^2', 'F770W/F115W^2', 'F444W/F410M^2', 'F444W/F356W^2', 'F444W/F277W^2', 'F444W/F200W^2', 'F444W/F150W^2', 'F444W/F115W^2', 'F410M/F356W^2', 'F410M/F277W^2', 'F410M/F200W^2', 'F410M/F150W^2', 'F410M/F115W^2', 'F356W/F277W^2', 'F356W/F200W^2', 'F356W/F150W^2', 'F356W/F115W^2', 'F277W/F200W^2', 'F277W/F150W^2', 'F277W/F115W^2', 'F200W/F150W^2', 'F200W/F115W^2', 'F150W/F115W^2']

   0%|          | 00:00/10:00

4.3 seconds
[I 2025-05-31 21:59:43,603] Trial 0 finished with value: -538.3346859999999 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 2.5396329035358637e-05, 'min_child_weight': 17, 'lambda': 25.752571083027494, 'device': 'cpu'}. Best is trial 0 with value: -538.3346859999999.
1.4 seconds
[I 2025-05-31 21:59:45,029] Trial 1 finished with value: -233.0358885 and parameters: {'stabilization': 'L2', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 6, 'gamma': 0.009805474860380612, 'min_child_weight': 222, 'lambda': 36.97821636577757, 'device': 'cpu'}. Best is trial 0 with value: -538.3346859999999.
2.0 seconds
[I 2025-05-31 21:59:47,073] Trial 2 finished with value: -568.1241150000001 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 9, 'gamma': 1.4233405719313036e-08, 'min_child_weight': 156, 'lambda': 5.785645476080961, 'device': 'cpu'}. Best is trial 2 with value: -568.1241150000001.
1.7 seconds
[I 2025-05-31 21:59:48,775] Trial 3 finished with value: -528.4065095000001 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 7, 'gamma': 0.41164506099767684, 'min_child_weight': 37, 'lambda': 4.999444210250469, 'device': 'cpu'}. Best is trial 2 with value: -568.1241150000001.
2.0 seconds
[I 2025-05-31 21:59:50,748] Trial 4 finished with value: -539.612793 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 5, 'gamma': 0.029989033512431876, 'min_child_weight': 72, 'lambda': 4.801093482603916, 'device': 'cpu'}. Best is trial 2 with value: -568.1241150000001.

invalid value encountered in subtract

1.9 seconds
[I 2025-05-31 21:59:52,643] Trial 5 finished with value: 100000000.0 and parameters: {'stabilization': 'MAD', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 4, 'gamma': 4.7025417911805535e-05, 'min_child_weight': 80, 'lambda': 3.4752834971002446, 'device': 'cpu'}. Best is trial 2 with value: -568.1241150000001.
1.9 seconds
[I 2025-05-31 21:59:54,506] Trial 6 finished with value: -440.689758 and parameters: {'stabilization': 'L2', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 6, 'gamma': 1.7553267516359528e-05, 'min_child_weight': 80, 'lambda': 50.83870744181027, 'device': 'cpu'}. Best is trial 2 with value: -568.1241150000001.
2.1 seconds
[I 2025-05-31 21:59:56,626] Trial 7 finished with value: -571.186035 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 6, 'gamma': 1.5543187142694343e-07, 'min_child_weight': 134, 'lambda': 7.636122743857147, 'device': 'cpu'}. Best is trial 7 with value: -571.186035.
1.7 seconds
[I 2025-05-31 21:59:58,351] Trial 8 finished with value: -414.29258749999997 and parameters: {'stabilization': 'L2', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 5, 'gamma': 3.7930128620486294, 'min_child_weight': 84, 'lambda': 9.221570011904374, 'device': 'cpu'}. Best is trial 7 with value: -571.186035.
1.9 seconds
[I 2025-05-31 22:00:00,224] Trial 9 finished with value: -550.360321 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 6, 'gamma': 0.016226422714380257, 'min_child_weight': 22, 'lambda': 5.7225183929734404, 'device': 'cpu'}. Best is trial 7 with value: -571.186035.
2.2 seconds
[I 2025-05-31 22:00:02,403] Trial 10 finished with value: -537.8868104999999 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 4, 'gamma': 1.7940320408573984e-06, 'min_child_weight': 219, 'lambda': 5.496936537996518, 'device': 'cpu'}. Best is trial 7 with value: -571.186035.
2.2 seconds
[I 2025-05-31 22:00:04,642] Trial 11 finished with value: -511.49543750000004 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 5, 'gamma': 1.5299822988853193e-08, 'min_child_weight': 58, 'lambda': 22.918740299170256, 'device': 'cpu'}. Best is trial 7 with value: -571.186035.
1.8 seconds
[I 2025-05-31 22:00:06,451] Trial 12 finished with value: -536.672455 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 9, 'gamma': 1.0310454091770936e-08, 'min_child_weight': 218, 'lambda': 1.2753821787593513, 'device': 'cpu'}. Best is trial 7 with value: -571.186035.
[I 2025-05-31 22:00:07,085] Trial 13 pruned. Trial was pruned at iteration 20.
1.7 seconds
[I 2025-05-31 22:00:08,845] Trial 14 finished with value: -549.9922489999999 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 6, 'gamma': 5.203454941578109e-08, 'min_child_weight': 179, 'lambda': 28.40801714583442, 'device': 'cpu'}. Best is trial 7 with value: -571.186035.
2.1 seconds
[I 2025-05-31 22:00:10,978] Trial 15 finished with value: -552.591797 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 9, 'gamma': 1.5029511921947606e-06, 'min_child_weight': 136, 'lambda': 8.546611186771843, 'device': 'cpu'}. Best is trial 7 with value: -571.186035.
[I 2025-05-31 22:00:11,651] Trial 16 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:00:12,350] Trial 17 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:00:13,140] Trial 18 pruned. Trial was pruned at iteration 20.
1.8 seconds
[I 2025-05-31 22:00:14,967] Trial 19 finished with value: -506.10517849999997 and parameters: {'stabilization': 'L2', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 6, 'gamma': 2.712873962951033e-07, 'min_child_weight': 53, 'lambda': 2.6021551800638707, 'device': 'cpu'}. Best is trial 7 with value: -571.186035.
1.9 seconds
[I 2025-05-31 22:00:16,880] Trial 20 finished with value: -536.210846 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 7, 'gamma': 1.1551116608310091e-06, 'min_child_weight': 114, 'lambda': 2.192434050611229, 'device': 'cpu'}. Best is trial 7 with value: -571.186035.
2.1 seconds
[I 2025-05-31 22:00:19,026] Trial 21 finished with value: -532.755478 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 6.531527613555491e-05, 'min_child_weight': 124, 'lambda': 5.228658688245536, 'device': 'cpu'}. Best is trial 7 with value: -571.186035.
2.1 seconds
[I 2025-05-31 22:00:21,112] Trial 22 finished with value: -552.2903745000001 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 8, 'gamma': 2.206230461601861e-08, 'min_child_weight': 132, 'lambda': 10.879565933655973, 'device': 'cpu'}. Best is trial 7 with value: -571.186035.
1.8 seconds
[I 2025-05-31 22:00:22,958] Trial 23 finished with value: -528.7008364999999 and parameters: {'stabilization': 'MAD', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 6, 'gamma': 4.636555608485337e-06, 'min_child_weight': 158, 'lambda': 70.11447167065849, 'device': 'cpu'}. Best is trial 7 with value: -571.186035.
[I 2025-05-31 22:00:23,565] Trial 24 pruned. Trial was pruned at iteration 20.
2.3 seconds
[I 2025-05-31 22:00:25,880] Trial 25 finished with value: -565.2005005 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 9, 'gamma': 3.3097555524427615e-08, 'min_child_weight': 103, 'lambda': 7.4590062241080135, 'device': 'cpu'}. Best is trial 7 with value: -571.186035.
3.4 seconds
[I 2025-05-31 22:00:29,298] Trial 26 finished with value: -536.43573 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 10, 'gamma': 4.725100012205021e-08, 'min_child_weight': 28, 'lambda': 16.02532558151139, 'device': 'cpu'}. Best is trial 7 with value: -571.186035.
2.5 seconds
[I 2025-05-31 22:00:31,767] Trial 27 finished with value: -557.6807859999999 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 10, 'gamma': 4.1196652381410564e-06, 'min_child_weight': 133, 'lambda': 6.635485580564186, 'device': 'cpu'}. Best is trial 7 with value: -571.186035.
2.7 seconds
[I 2025-05-31 22:00:34,476] Trial 28 finished with value: -526.4791565 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 10, 'gamma': 2.1382968788984822e-07, 'min_child_weight': 87, 'lambda': 20.566345108533405, 'device': 'cpu'}. Best is trial 7 with value: -571.186035.
[I 2025-05-31 22:00:35,195] Trial 29 pruned. Trial was pruned at iteration 20.
1.9 seconds
[I 2025-05-31 22:00:37,120] Trial 30 finished with value: -559.1357725 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 10, 'gamma': 1.7756430613316088e-08, 'min_child_weight': 149, 'lambda': 8.275129902685977, 'device': 'cpu'}. Best is trial 7 with value: -571.186035.
1.8 seconds
[I 2025-05-31 22:00:38,904] Trial 31 finished with value: -566.6150514999999 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 9, 'gamma': 1.2463050681972017e-08, 'min_child_weight': 146, 'lambda': 9.775925995982428, 'device': 'cpu'}. Best is trial 7 with value: -571.186035.
1.9 seconds
[I 2025-05-31 22:00:40,806] Trial 32 finished with value: -560.3184205 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 8, 'gamma': 1.4046435953988977e-08, 'min_child_weight': 129, 'lambda': 9.162166516737194, 'device': 'cpu'}. Best is trial 7 with value: -571.186035.
[I 2025-05-31 22:00:41,397] Trial 33 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:00:41,965] Trial 34 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:00:42,633] Trial 35 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:00:43,235] Trial 36 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:00:43,886] Trial 37 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:00:44,462] Trial 38 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:00:45,106] Trial 39 pruned. Trial was pruned at iteration 20.
2.2 seconds
[I 2025-05-31 22:00:47,300] Trial 40 finished with value: -571.6891175000001 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 6, 'gamma': 7.860624643290146e-06, 'min_child_weight': 108, 'lambda': 9.032580896099716, 'device': 'cpu'}. Best is trial 40 with value: -571.6891175000001.

invalid value encountered in add

[I 2025-05-31 22:00:49,287] Trial 41 pruned. Trial was pruned at iteration 125.
[I 2025-05-31 22:00:50,026] Trial 42 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:00:50,761] Trial 43 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:00:52,781] Trial 44 pruned. Trial was pruned at iteration 125.
[I 2025-05-31 22:00:53,367] Trial 45 pruned. Trial was pruned at iteration 20.
1.8 seconds
[I 2025-05-31 22:00:55,195] Trial 46 finished with value: -565.3455505 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 4, 'gamma': 1.8679492218445252e-07, 'min_child_weight': 126, 'lambda': 3.4176836684664607, 'device': 'cpu'}. Best is trial 40 with value: -571.6891175000001.
2.1 seconds
[I 2025-05-31 22:00:57,349] Trial 47 finished with value: -550.0536195 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 5, 'gamma': 5.007938682813857e-08, 'min_child_weight': 122, 'lambda': 5.789844346969788, 'device': 'cpu'}. Best is trial 40 with value: -571.6891175000001.
1.6 seconds
[I 2025-05-31 22:00:59,003] Trial 48 finished with value: -551.1981505 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 3, 'gamma': 2.9976062365699335e-05, 'min_child_weight': 116, 'lambda': 1.8864465740640275, 'device': 'cpu'}. Best is trial 40 with value: -571.6891175000001.
1.9 seconds
[I 2025-05-31 22:01:00,896] Trial 49 finished with value: -559.3542175 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 4, 'gamma': 1.1595978789306633e-06, 'min_child_weight': 157, 'lambda': 1.7606221450790216, 'device': 'cpu'}. Best is trial 40 with value: -571.6891175000001.
[I 2025-05-31 22:01:01,477] Trial 50 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:01:02,069] Trial 51 pruned. Trial was pruned at iteration 20.
2.0 seconds
[I 2025-05-31 22:01:04,052] Trial 52 finished with value: -544.5730590000001 and parameters: {'stabilization': 'L2', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 5, 'gamma': 0.013184669771200148, 'min_child_weight': 13, 'lambda': 11.86539041879886, 'device': 'cpu'}. Best is trial 40 with value: -571.6891175000001.
1.7 seconds
[I 2025-05-31 22:01:05,747] Trial 53 finished with value: -551.305145 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 8, 'gamma': 9.953853208798318e-08, 'min_child_weight': 227, 'lambda': 12.166813817434917, 'device': 'cpu'}. Best is trial 40 with value: -571.6891175000001.
1.8 seconds
[I 2025-05-31 22:01:07,510] Trial 54 finished with value: -537.8863220000001 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 4, 'gamma': 5.0711814335032386e-05, 'min_child_weight': 104, 'lambda': 12.41428108353893, 'device': 'cpu'}. Best is trial 40 with value: -571.6891175000001.
1.6 seconds
[I 2025-05-31 22:01:09,080] Trial 55 finished with value: -563.784027 and parameters: {'stabilization': 'MAD', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 3, 'gamma': 0.0015880752905682901, 'min_child_weight': 79, 'lambda': 119.42356346215409, 'device': 'cpu'}. Best is trial 40 with value: -571.6891175000001.
[I 2025-05-31 22:01:09,720] Trial 56 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:01:10,380] Trial 57 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:01:11,014] Trial 58 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:01:11,629] Trial 59 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:01:12,209] Trial 60 pruned. Trial was pruned at iteration 20.
1.6 seconds
[I 2025-05-31 22:01:13,805] Trial 61 finished with value: -538.716034 and parameters: {'stabilization': 'MAD', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 3, 'gamma': 0.0012966855940974174, 'min_child_weight': 65, 'lambda': 146.2106582631281, 'device': 'cpu'}. Best is trial 40 with value: -571.6891175000001.
1.6 seconds
[I 2025-05-31 22:01:15,374] Trial 62 finished with value: -529.4952695 and parameters: {'stabilization': 'MAD', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 4, 'gamma': 0.03248268866167507, 'min_child_weight': 169, 'lambda': 86.1387905861123, 'device': 'cpu'}. Best is trial 40 with value: -571.6891175000001.

invalid value encountered in subtract
invalid value encountered in add

2.1 seconds
[I 2025-05-31 22:01:17,491] Trial 63 finished with value: 100000000.0 and parameters: {'stabilization': 'MAD', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 3, 'gamma': 4.82218843841422, 'min_child_weight': 97, 'lambda': 46.86953830724628, 'device': 'cpu'}. Best is trial 40 with value: -571.6891175000001.
[I 2025-05-31 22:01:18,141] Trial 64 pruned. Trial was pruned at iteration 20.

invalid value encountered in subtract

2.1 seconds
[I 2025-05-31 22:01:20,215] Trial 65 finished with value: 100000000.0 and parameters: {'stabilization': 'MAD', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 3, 'gamma': 0.000131061786050065, 'min_child_weight': 68, 'lambda': 21.07070990752435, 'device': 'cpu'}. Best is trial 40 with value: -571.6891175000001.
1.9 seconds
[I 2025-05-31 22:01:22,081] Trial 66 finished with value: -554.254608 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 5, 'gamma': 7.936866283744524e-07, 'min_child_weight': 70, 'lambda': 24.188763934320402, 'device': 'cpu'}. Best is trial 40 with value: -571.6891175000001.

invalid value encountered in subtract

2.2 seconds
[I 2025-05-31 22:01:24,331] Trial 67 finished with value: 100000000.0 and parameters: {'stabilization': 'MAD', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 3, 'gamma': 0.02637272250292243, 'min_child_weight': 75, 'lambda': 70.54541585403223, 'device': 'cpu'}. Best is trial 40 with value: -571.6891175000001.
[I 2025-05-31 22:01:24,915] Trial 68 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:01:25,619] Trial 69 pruned. Trial was pruned at iteration 20.
1.8 seconds
[I 2025-05-31 22:01:27,436] Trial 70 finished with value: -572.7865294999999 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 3, 'gamma': 0.007191774416874417, 'min_child_weight': 96, 'lambda': 122.73669366406183, 'device': 'cpu'}. Best is trial 70 with value: -572.7865294999999.
1.8 seconds
[I 2025-05-31 22:01:29,235] Trial 71 finished with value: -534.8459625 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 4, 'gamma': 0.00034089888889644523, 'min_child_weight': 124, 'lambda': 41.1675794109943, 'device': 'cpu'}. Best is trial 70 with value: -572.7865294999999.
1.7 seconds
[I 2025-05-31 22:01:30,930] Trial 72 finished with value: -530.1312869999999 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 3, 'gamma': 0.11517957680592521, 'min_child_weight': 51, 'lambda': 105.83644005065398, 'device': 'cpu'}. Best is trial 70 with value: -572.7865294999999.
1.8 seconds
[I 2025-05-31 22:01:32,695] Trial 73 finished with value: -545.870453 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 3, 'gamma': 1.1231274710787972e-07, 'min_child_weight': 58, 'lambda': 2.042495528112113, 'device': 'cpu'}. Best is trial 70 with value: -572.7865294999999.
[I 2025-05-31 22:01:33,405] Trial 74 pruned. Trial was pruned at iteration 20.
2.0 seconds
[I 2025-05-31 22:01:35,384] Trial 75 finished with value: -544.8724669999999 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 4, 'gamma': 0.00041443077655468804, 'min_child_weight': 49, 'lambda': 141.06296878866524, 'device': 'cpu'}. Best is trial 70 with value: -572.7865294999999.
[I 2025-05-31 22:01:36,010] Trial 76 pruned. Trial was pruned at iteration 20.
1.7 seconds
[I 2025-05-31 22:01:37,768] Trial 77 finished with value: -545.0274655000001 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 9, 'gamma': 1.3723014280978176e-08, 'min_child_weight': 170, 'lambda': 4.881785497786435, 'device': 'cpu'}. Best is trial 70 with value: -572.7865294999999.
2.0 seconds
[I 2025-05-31 22:01:39,808] Trial 78 finished with value: -535.615631 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 8, 'gamma': 1.2973099002718484e-06, 'min_child_weight': 80, 'lambda': 16.370492551210816, 'device': 'cpu'}. Best is trial 70 with value: -572.7865294999999.
[I 2025-05-31 22:01:40,523] Trial 79 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:01:41,151] Trial 80 pruned. Trial was pruned at iteration 20.
2.0 seconds
[I 2025-05-31 22:01:43,166] Trial 81 finished with value: -555.6707155 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 9, 'gamma': 1.3541146314592335e-08, 'min_child_weight': 98, 'lambda': 12.916317472553699, 'device': 'cpu'}. Best is trial 70 with value: -572.7865294999999.
[I 2025-05-31 22:01:43,820] Trial 82 pruned. Trial was pruned at iteration 20.
1.9 seconds
[I 2025-05-31 22:01:45,756] Trial 83 finished with value: -549.8399045 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 7, 'gamma': 3.7625956186465914e-08, 'min_child_weight': 118, 'lambda': 7.829757116547688, 'device': 'cpu'}. Best is trial 70 with value: -572.7865294999999.
1.7 seconds
[I 2025-05-31 22:01:47,439] Trial 84 finished with value: -535.5687865 and parameters: {'stabilization': 'L2', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 5, 'gamma': 0.0056184079927858325, 'min_child_weight': 43, 'lambda': 1.2266838607730863, 'device': 'cpu'}. Best is trial 70 with value: -572.7865294999999.
[I 2025-05-31 22:01:48,106] Trial 85 pruned. Trial was pruned at iteration 20.
1.8 seconds
[I 2025-05-31 22:01:49,947] Trial 86 finished with value: -556.758148 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 3, 'gamma': 0.021367393522320883, 'min_child_weight': 161, 'lambda': 134.5369432225549, 'device': 'cpu'}. Best is trial 70 with value: -572.7865294999999.
1.8 seconds
[I 2025-05-31 22:01:51,782] Trial 87 finished with value: -538.9547729999999 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 8, 'gamma': 5.573561560266356e-07, 'min_child_weight': 135, 'lambda': 52.149646051217054, 'device': 'cpu'}. Best is trial 70 with value: -572.7865294999999.
[I 2025-05-31 22:01:52,519] Trial 88 pruned. Trial was pruned at iteration 20.
1.9 seconds
[I 2025-05-31 22:01:54,465] Trial 89 finished with value: -543.941101 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 6, 'gamma': 1.6879980509145527e-07, 'min_child_weight': 202, 'lambda': 4.764289713123308, 'device': 'cpu'}. Best is trial 70 with value: -572.7865294999999.
2.1 seconds
[I 2025-05-31 22:01:56,600] Trial 90 finished with value: -561.8856505 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 7, 'gamma': 1.282766479899665, 'min_child_weight': 91, 'lambda': 95.1504140205716, 'device': 'cpu'}. Best is trial 70 with value: -572.7865294999999.
1.9 seconds
[I 2025-05-31 22:01:58,544] Trial 91 finished with value: -577.146088 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 9, 'gamma': 0.15521317103010426, 'min_child_weight': 108, 'lambda': 113.6533104326656, 'device': 'cpu'}. Best is trial 91 with value: -577.146088.
[I 2025-05-31 22:01:59,187] Trial 92 pruned. Trial was pruned at iteration 20.
1.8 seconds
[I 2025-05-31 22:02:01,038] Trial 93 finished with value: -569.622406 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 9, 'gamma': 5.054061246489338, 'min_child_weight': 140, 'lambda': 83.43365168986566, 'device': 'cpu'}. Best is trial 91 with value: -577.146088.
2.1 seconds
[I 2025-05-31 22:02:03,107] Trial 94 finished with value: -557.7799685 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 10, 'gamma': 0.05953051769941643, 'min_child_weight': 81, 'lambda': 66.51962804228228, 'device': 'cpu'}. Best is trial 91 with value: -577.146088.
[I 2025-05-31 22:02:03,719] Trial 95 pruned. Trial was pruned at iteration 20.
1.9 seconds
[I 2025-05-31 22:02:05,590] Trial 96 finished with value: -550.1198119999999 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 5.916949492530223, 'min_child_weight': 168, 'lambda': 96.54305369005682, 'device': 'cpu'}. Best is trial 91 with value: -577.146088.
1.9 seconds
[I 2025-05-31 22:02:07,479] Trial 97 finished with value: -569.898254 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 9, 'gamma': 0.7867385174584295, 'min_child_weight': 141, 'lambda': 88.73247867666245, 'device': 'cpu'}. Best is trial 91 with value: -577.146088.
2.0 seconds
[I 2025-05-31 22:02:09,506] Trial 98 finished with value: -560.8153995 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 9, 'gamma': 2.013476162679143, 'min_child_weight': 174, 'lambda': 73.87906293203815, 'device': 'cpu'}. Best is trial 91 with value: -577.146088.
[I 2025-05-31 22:02:10,100] Trial 99 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:02:10,793] Trial 100 pruned. Trial was pruned at iteration 20.
1.9 seconds
[I 2025-05-31 22:02:12,662] Trial 101 finished with value: -576.359009 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 9, 'gamma': 0.03939823179896599, 'min_child_weight': 129, 'lambda': 98.55138888202369, 'device': 'cpu'}. Best is trial 91 with value: -577.146088.
[I 2025-05-31 22:02:13,280] Trial 102 pruned. Trial was pruned at iteration 20.
1.9 seconds
[I 2025-05-31 22:02:15,154] Trial 103 finished with value: -558.8230590000001 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 7, 'gamma': 0.00710884873690168, 'min_child_weight': 172, 'lambda': 126.3949915037119, 'device': 'cpu'}. Best is trial 91 with value: -577.146088.
1.9 seconds
[I 2025-05-31 22:02:17,081] Trial 104 finished with value: -585.4270630000001 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 9, 'gamma': 0.0019938022688501832, 'min_child_weight': 105, 'lambda': 70.07508991962831, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
2.2 seconds
[I 2025-05-31 22:02:19,262] Trial 105 finished with value: -565.6244815 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 8, 'gamma': 0.0014696837690568924, 'min_child_weight': 90, 'lambda': 112.37693943301231, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
2.1 seconds
[I 2025-05-31 22:02:21,339] Trial 106 finished with value: -566.5224915 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 8, 'gamma': 8.068991626456967e-05, 'min_child_weight': 86, 'lambda': 120.32719277321299, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
1.9 seconds
[I 2025-05-31 22:02:23,233] Trial 107 finished with value: -565.115906 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 10, 'gamma': 0.09390706538319665, 'min_child_weight': 141, 'lambda': 133.79327025180092, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
[I 2025-05-31 22:02:23,854] Trial 108 pruned. Trial was pruned at iteration 20.
1.9 seconds
[I 2025-05-31 22:02:25,722] Trial 109 finished with value: -543.6245115 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 10, 'gamma': 0.0021618337111951273, 'min_child_weight': 164, 'lambda': 24.877169306110012, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
1.6 seconds
[I 2025-05-31 22:02:27,348] Trial 110 finished with value: -556.3331605000001 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 8, 'gamma': 8.023579739007078, 'min_child_weight': 98, 'lambda': 26.055687660008193, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
2.0 seconds
[I 2025-05-31 22:02:29,323] Trial 111 finished with value: -568.5903625 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 8, 'gamma': 0.020213745529973536, 'min_child_weight': 102, 'lambda': 70.49304036914943, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
2.3 seconds
[I 2025-05-31 22:02:31,681] Trial 112 finished with value: -571.2257079999999 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 7, 'gamma': 0.00046499685552308344, 'min_child_weight': 94, 'lambda': 149.3303661556515, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
2.7 seconds
[I 2025-05-31 22:02:34,369] Trial 113 finished with value: -553.1926269999999 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 7, 'gamma': 0.00021643442379436624, 'min_child_weight': 69, 'lambda': 71.98310710317897, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
[I 2025-05-31 22:02:34,994] Trial 114 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:02:35,660] Trial 115 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:02:36,342] Trial 116 pruned. Trial was pruned at iteration 20.
2.3 seconds
[I 2025-05-31 22:02:38,641] Trial 117 finished with value: -570.8308105 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 8, 'gamma': 0.0035137973294394166, 'min_child_weight': 134, 'lambda': 127.40266355787772, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
2.4 seconds
[I 2025-05-31 22:02:41,013] Trial 118 finished with value: -560.286987 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 0.00783175568676248, 'min_child_weight': 79, 'lambda': 111.96176947563438, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
2.0 seconds
[I 2025-05-31 22:02:43,028] Trial 119 finished with value: -564.5676880000001 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 7, 'gamma': 0.0007596093189488856, 'min_child_weight': 157, 'lambda': 131.70317411761158, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
2.2 seconds
[I 2025-05-31 22:02:45,220] Trial 120 finished with value: -570.505127 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 7, 'gamma': 3.3569837959173813e-07, 'min_child_weight': 107, 'lambda': 116.93883747624722, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
[I 2025-05-31 22:02:45,811] Trial 121 pruned. Trial was pruned at iteration 20.
1.6 seconds
[I 2025-05-31 22:02:47,465] Trial 122 finished with value: -567.2421265 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 9, 'gamma': 7.5564354921252495, 'min_child_weight': 121, 'lambda': 106.18723481429146, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
2.1 seconds
[I 2025-05-31 22:02:49,574] Trial 123 finished with value: -555.3678285000001 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 7, 'gamma': 5.390663400151784e-08, 'min_child_weight': 106, 'lambda': 72.9932561448359, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
[I 2025-05-31 22:02:50,173] Trial 124 pruned. Trial was pruned at iteration 20.
2.4 seconds
[I 2025-05-31 22:02:52,623] Trial 125 finished with value: -566.4755555 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 8, 'gamma': 0.04263672929306466, 'min_child_weight': 88, 'lambda': 117.40530076550712, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
2.1 seconds
[I 2025-05-31 22:02:54,762] Trial 126 finished with value: -554.646027 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 5, 'gamma': 0.0007435140429590045, 'min_child_weight': 89, 'lambda': 147.96834496006034, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
[I 2025-05-31 22:02:55,358] Trial 127 pruned. Trial was pruned at iteration 20.
2.5 seconds
[I 2025-05-31 22:02:57,861] Trial 128 finished with value: -558.957367 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 7, 'gamma': 1.3912651924774933e-07, 'min_child_weight': 106, 'lambda': 9.976358215091079, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
[I 2025-05-31 22:02:58,505] Trial 129 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:02:59,109] Trial 130 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:02:59,729] Trial 131 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:03:00,342] Trial 132 pruned. Trial was pruned at iteration 20.

invalid value encountered in subtract

2.5 seconds
[I 2025-05-31 22:03:02,860] Trial 133 finished with value: 100000000.0 and parameters: {'stabilization': 'MAD', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 7, 'gamma': 8.52159439565328e-08, 'min_child_weight': 53, 'lambda': 97.20698026222445, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
[I 2025-05-31 22:03:03,501] Trial 134 pruned. Trial was pruned at iteration 20.
1.9 seconds
[I 2025-05-31 22:03:05,455] Trial 135 finished with value: -564.3602295 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 8, 'gamma': 0.983718521423068, 'min_child_weight': 115, 'lambda': 126.82858131454422, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
[I 2025-05-31 22:03:06,076] Trial 136 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:03:06,775] Trial 137 pruned. Trial was pruned at iteration 20.
1.5 seconds
[I 2025-05-31 22:03:08,281] Trial 138 finished with value: -562.680664 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 10, 'gamma': 15.019922925552814, 'min_child_weight': 127, 'lambda': 25.365996026225513, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
1.7 seconds
[I 2025-05-31 22:03:09,949] Trial 139 finished with value: -569.9113769999999 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 9, 'gamma': 9.501344390986599, 'min_child_weight': 90, 'lambda': 65.7399190186448, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
[I 2025-05-31 22:03:10,623] Trial 140 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:03:11,252] Trial 141 pruned. Trial was pruned at iteration 20.
2.0 seconds
[I 2025-05-31 22:03:13,230] Trial 142 finished with value: -574.0078430000001 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 9, 'gamma': 2.3132838969836595, 'min_child_weight': 132, 'lambda': 130.66472371122643, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
2.0 seconds
[I 2025-05-31 22:03:15,248] Trial 143 finished with value: -579.5551755 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 8, 'gamma': 0.016202155007680775, 'min_child_weight': 96, 'lambda': 38.935427269526215, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
2.3 seconds
[I 2025-05-31 22:03:17,537] Trial 144 finished with value: -562.923828 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 8, 'gamma': 2.770545064265404e-06, 'min_child_weight': 94, 'lambda': 108.30594390272606, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
1.9 seconds
[I 2025-05-31 22:03:19,451] Trial 145 finished with value: -557.0128784999999 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 9, 'gamma': 0.07305821559996975, 'min_child_weight': 99, 'lambda': 36.7362950282458, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
1.7 seconds
[I 2025-05-31 22:03:21,189] Trial 146 finished with value: -548.016266 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 9, 'gamma': 0.0004368849395024274, 'min_child_weight': 171, 'lambda': 102.35060705640052, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
2.8 seconds
[I 2025-05-31 22:03:23,965] Trial 147 finished with value: -555.720581 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 6, 'gamma': 8.157899770713593e-08, 'min_child_weight': 29, 'lambda': 144.48042547089233, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
2.2 seconds
[I 2025-05-31 22:03:26,146] Trial 148 finished with value: -583.9998475 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 6, 'gamma': 7.025135466448473e-05, 'min_child_weight': 72, 'lambda': 13.639521462455042, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
[I 2025-05-31 22:03:26,852] Trial 149 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:03:27,454] Trial 150 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:03:28,306] Trial 151 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:03:28,898] Trial 152 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:03:29,566] Trial 153 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:03:30,272] Trial 154 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:03:30,936] Trial 155 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:03:31,518] Trial 156 pruned. Trial was pruned at iteration 20.
2.1 seconds
[I 2025-05-31 22:03:33,668] Trial 157 finished with value: -562.604645 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 10, 'gamma': 8.263882864616723e-05, 'min_child_weight': 72, 'lambda': 39.75121615604367, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
2.3 seconds
[I 2025-05-31 22:03:35,956] Trial 158 finished with value: -551.5425415 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 7, 'gamma': 0.000401529310553666, 'min_child_weight': 78, 'lambda': 12.51160913599006, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
1.7 seconds
[I 2025-05-31 22:03:37,631] Trial 159 finished with value: -571.4435424999999 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 3, 'gamma': 0.001999028217939728, 'min_child_weight': 96, 'lambda': 52.55218079393602, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
1.9 seconds
[I 2025-05-31 22:03:39,533] Trial 160 finished with value: -557.6372985 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 6, 'gamma': 5.575523471135619e-07, 'min_child_weight': 133, 'lambda': 8.353739646631562, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
1.7 seconds
[I 2025-05-31 22:03:41,267] Trial 161 finished with value: -561.934113 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 4, 'gamma': 0.00905510004166771, 'min_child_weight': 132, 'lambda': 35.76373336616059, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
1.9 seconds
[I 2025-05-31 22:03:43,209] Trial 162 finished with value: -557.813263 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 3, 'gamma': 0.0065216525836809145, 'min_child_weight': 108, 'lambda': 93.22615130610858, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
[I 2025-05-31 22:03:43,792] Trial 163 pruned. Trial was pruned at iteration 20.
2.3 seconds
[I 2025-05-31 22:03:46,090] Trial 164 finished with value: -562.4273985 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 9, 'gamma': 0.03436998540917756, 'min_child_weight': 81, 'lambda': 78.55235813813921, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
[I 2025-05-31 22:03:46,670] Trial 165 pruned. Trial was pruned at iteration 20.
1.9 seconds
[I 2025-05-31 22:03:48,596] Trial 166 finished with value: -566.3221435 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 3, 'gamma': 0.6086420759927776, 'min_child_weight': 93, 'lambda': 94.82010597823499, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
1.7 seconds
[I 2025-05-31 22:03:50,337] Trial 167 finished with value: -546.8621215000001 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 3, 'gamma': 0.0004712016445965929, 'min_child_weight': 53, 'lambda': 39.902940772667364, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
2.1 seconds
[I 2025-05-31 22:03:52,472] Trial 168 finished with value: -577.0916745 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 7, 'gamma': 0.002911621446984091, 'min_child_weight': 121, 'lambda': 144.34064380260932, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
1.8 seconds
[I 2025-05-31 22:03:54,325] Trial 169 finished with value: -554.0499875 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 8, 'gamma': 0.8925027982061278, 'min_child_weight': 160, 'lambda': 138.05090325644838, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
2.0 seconds
[I 2025-05-31 22:03:56,383] Trial 170 finished with value: -562.6233215 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 6, 'gamma': 0.01799333571184205, 'min_child_weight': 135, 'lambda': 145.35636465334727, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
2.0 seconds
[I 2025-05-31 22:03:58,412] Trial 171 finished with value: -574.9880674999999 and parameters: {'stabilization': 'MAD', 'response_fn': 'softplus', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 9, 'gamma': 0.12754188516895057, 'min_child_weight': 127, 'lambda': 117.71054455322371, 'device': 'cpu'}. Best is trial 104 with value: -585.4270630000001.
[I 2025-05-31 22:03:59,055] Trial 172 pruned. Trial was pruned at iteration 20.
2.5 seconds
[I 2025-05-31 22:04:01,590] Trial 173 finished with value: -593.6697995 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 0.005434830633400749, 'min_child_weight': 33, 'lambda': 119.55250590983898, 'device': 'cpu'}. Best is trial 173 with value: -593.6697995.
2.4 seconds
[I 2025-05-31 22:04:04,040] Trial 174 finished with value: -588.3483275 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 9, 'gamma': 0.006448513670224484, 'min_child_weight': 25, 'lambda': 59.98751721071317, 'device': 'cpu'}. Best is trial 173 with value: -593.6697995.
2.4 seconds
[I 2025-05-31 22:04:06,437] Trial 175 finished with value: -565.9078065 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 9, 'gamma': 0.006973605271577416, 'min_child_weight': 30, 'lambda': 25.26985876788745, 'device': 'cpu'}. Best is trial 173 with value: -593.6697995.
2.7 seconds
[I 2025-05-31 22:04:09,165] Trial 176 finished with value: -587.1781315000001 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 0.0014749625770416434, 'min_child_weight': 10, 'lambda': 30.063040255345115, 'device': 'cpu'}. Best is trial 173 with value: -593.6697995.
1.8 seconds
[I 2025-05-31 22:04:10,967] Trial 177 finished with value: -586.3135374999999 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 0.8595646128529425, 'min_child_weight': 32, 'lambda': 122.31996900370613, 'device': 'cpu'}. Best is trial 173 with value: -593.6697995.
2.3 seconds
[I 2025-05-31 22:04:13,241] Trial 178 finished with value: -580.9576415 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 9, 'gamma': 0.020901799267839614, 'min_child_weight': 35, 'lambda': 73.6979159455691, 'device': 'cpu'}. Best is trial 173 with value: -593.6697995.
1.7 seconds
[I 2025-05-31 22:04:15,005] Trial 179 finished with value: -585.6549685 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 9, 'gamma': 1.1299263110621307, 'min_child_weight': 20, 'lambda': 129.79762970243408, 'device': 'cpu'}. Best is trial 173 with value: -593.6697995.
[I 2025-05-31 22:04:15,633] Trial 180 pruned. Trial was pruned at iteration 20.
2.4 seconds
[I 2025-05-31 22:04:18,011] Trial 181 finished with value: -565.0000915 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 9, 'gamma': 0.03946945825129624, 'min_child_weight': 4, 'lambda': 90.25280271421961, 'device': 'cpu'}. Best is trial 173 with value: -593.6697995.
[I 2025-05-31 22:04:18,643] Trial 182 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:04:19,522] Trial 183 pruned. Trial was pruned at iteration 20.
2.5 seconds
[I 2025-05-31 22:04:22,078] Trial 184 finished with value: -597.4772945 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 9.596151705181202e-05, 'min_child_weight': 31, 'lambda': 98.78838252818872, 'device': 'cpu'}. Best is trial 184 with value: -597.4772945.
2.3 seconds
[I 2025-05-31 22:04:24,407] Trial 185 finished with value: -575.258911 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 8, 'gamma': 2.909575521235058e-06, 'min_child_weight': 48, 'lambda': 44.8010066255139, 'device': 'cpu'}. Best is trial 184 with value: -597.4772945.
[I 2025-05-31 22:04:25,072] Trial 186 pruned. Trial was pruned at iteration 20.
2.5 seconds
[I 2025-05-31 22:04:27,548] Trial 187 finished with value: -599.183777 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 9, 'gamma': 1.9111290329601907e-06, 'min_child_weight': 38, 'lambda': 142.4996020188469, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
2.5 seconds
[I 2025-05-31 22:04:30,052] Trial 188 finished with value: -581.9074705 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 4.1465277784778914e-05, 'min_child_weight': 36, 'lambda': 77.28320288251074, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
2.8 seconds
[I 2025-05-31 22:04:32,830] Trial 189 finished with value: -591.335327 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 1.309232502330243e-06, 'min_child_weight': 16, 'lambda': 110.55214791189547, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
2.9 seconds
[I 2025-05-31 22:04:35,741] Trial 190 finished with value: -592.5753784999999 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 7.591166070282343e-07, 'min_child_weight': 16, 'lambda': 108.35646225989339, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
2.8 seconds
[I 2025-05-31 22:04:38,544] Trial 191 finished with value: -582.3792725000001 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 7.631319873667642e-05, 'min_child_weight': 17, 'lambda': 128.64667335152433, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
[I 2025-05-31 22:04:39,241] Trial 192 pruned. Trial was pruned at iteration 20.
2.9 seconds
[I 2025-05-31 22:04:42,145] Trial 193 finished with value: -587.864807 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 9, 'gamma': 1.643805723692164e-05, 'min_child_weight': 8, 'lambda': 72.13446750665545, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
2.7 seconds
[I 2025-05-31 22:04:44,889] Trial 194 finished with value: -587.7263794999999 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 5.4181461931134994e-05, 'min_child_weight': 15, 'lambda': 79.05088236833085, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
2.3 seconds
[I 2025-05-31 22:04:47,195] Trial 195 finished with value: -587.3322145 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 10, 'gamma': 3.49591650400422e-05, 'min_child_weight': 12, 'lambda': 43.20545390995926, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
2.1 seconds
[I 2025-05-31 22:04:49,335] Trial 196 finished with value: -597.4509885 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 9, 'gamma': 0.0009592064618035505, 'min_child_weight': 36, 'lambda': 110.58807516276524, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
2.2 seconds
[I 2025-05-31 22:04:51,527] Trial 197 finished with value: -589.43396 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 10, 'gamma': 0.0024079239940583276, 'min_child_weight': 15, 'lambda': 61.72865332470975, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
2.4 seconds
[I 2025-05-31 22:04:53,934] Trial 198 finished with value: -577.5836795 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 9, 'gamma': 7.169945701767782e-05, 'min_child_weight': 3, 'lambda': 93.66505999871147, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
1.8 seconds
[I 2025-05-31 22:04:55,721] Trial 199 finished with value: -574.5710449999999 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 10, 'gamma': 0.16071475285654765, 'min_child_weight': 48, 'lambda': 54.13010926380267, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
[I 2025-05-31 22:04:56,402] Trial 200 pruned. Trial was pruned at iteration 20.
2.1 seconds
[I 2025-05-31 22:04:58,546] Trial 201 finished with value: -569.057373 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 9, 'gamma': 3.447688058528674e-05, 'min_child_weight': 38, 'lambda': 42.669568027841, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
2.6 seconds
[I 2025-05-31 22:05:01,193] Trial 202 finished with value: -582.401276 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 6.714145151965375e-05, 'min_child_weight': 24, 'lambda': 70.69393315516336, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
3.3 seconds
[I 2025-05-31 22:05:04,539] Trial 203 finished with value: -554.918579 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 7.89591472212701e-05, 'min_child_weight': 1, 'lambda': 120.43713376942627, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.

invalid value encountered in subtract

2.5 seconds
[I 2025-05-31 22:05:07,073] Trial 204 finished with value: 100000000.0 and parameters: {'stabilization': 'MAD', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 0.3144912311448094, 'min_child_weight': 64, 'lambda': 71.25598865752376, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
2.7 seconds
[I 2025-05-31 22:05:09,837] Trial 205 finished with value: -591.6864925 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 1.0389562372846309e-07, 'min_child_weight': 16, 'lambda': 92.9559870292283, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
[I 2025-05-31 22:05:10,498] Trial 206 pruned. Trial was pruned at iteration 20.
2.7 seconds
[I 2025-05-31 22:05:13,247] Trial 207 finished with value: -584.461151 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 2.085923081816322e-05, 'min_child_weight': 15, 'lambda': 22.586236104800115, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
[I 2025-05-31 22:05:14,115] Trial 208 pruned. Trial was pruned at iteration 20.
2.7 seconds
[I 2025-05-31 22:05:16,847] Trial 209 finished with value: -574.6616825000001 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 3.264390389517532e-05, 'min_child_weight': 11, 'lambda': 29.252036610466583, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
[I 2025-05-31 22:05:17,517] Trial 210 pruned. Trial was pruned at iteration 20.
2.5 seconds
[I 2025-05-31 22:05:20,008] Trial 211 finished with value: -589.223419 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 9, 'gamma': 2.2988261763199818e-08, 'min_child_weight': 31, 'lambda': 55.98258178202411, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
2.6 seconds
[I 2025-05-31 22:05:22,657] Trial 212 finished with value: -591.512787 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 9, 'gamma': 2.5245260125483864e-08, 'min_child_weight': 17, 'lambda': 58.518558530503384, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
2.5 seconds
[I 2025-05-31 22:05:25,126] Trial 213 finished with value: -565.959747 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 6, 'gamma': 1.6656091854178262e-08, 'min_child_weight': 4, 'lambda': 58.34355172862271, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
1.9 seconds
[I 2025-05-31 22:05:27,084] Trial 214 finished with value: -578.0919495 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 8, 'gamma': 0.00989673944193604, 'min_child_weight': 43, 'lambda': 92.80902615703727, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
2.8 seconds
[I 2025-05-31 22:05:29,948] Trial 215 finished with value: -578.4188839999999 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 9, 'gamma': 9.688911844423959e-08, 'min_child_weight': 7, 'lambda': 104.56682995875201, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
[I 2025-05-31 22:05:30,640] Trial 216 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:05:31,588] Trial 217 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:05:32,363] Trial 218 pruned. Trial was pruned at iteration 20.
2.7 seconds
[I 2025-05-31 22:05:35,075] Trial 219 finished with value: -565.917633 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 9, 'gamma': 5.362068500764623e-08, 'min_child_weight': 11, 'lambda': 26.88680742571276, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
2.7 seconds
[I 2025-05-31 22:05:37,766] Trial 220 finished with value: -589.1559145 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 2.0066212276628484e-08, 'min_child_weight': 30, 'lambda': 103.85920387144012, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
2.6 seconds
[I 2025-05-31 22:05:40,343] Trial 221 finished with value: -599.1741939999999 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 1.2415044674414208e-08, 'min_child_weight': 36, 'lambda': 125.28368131545064, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
[I 2025-05-31 22:05:41,007] Trial 222 pruned. Trial was pruned at iteration 20.
2.4 seconds
[I 2025-05-31 22:05:43,379] Trial 223 finished with value: -562.871185 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 10, 'gamma': 1.792701201817649e-05, 'min_child_weight': 4, 'lambda': 27.600013003315084, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
2.5 seconds
[I 2025-05-31 22:05:45,937] Trial 224 finished with value: -594.3693845 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 3.078892702151498e-07, 'min_child_weight': 39, 'lambda': 122.3906141198032, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
[I 2025-05-31 22:05:46,789] Trial 225 pruned. Trial was pruned at iteration 20.
2.7 seconds
[I 2025-05-31 22:05:49,509] Trial 226 finished with value: -594.8168335 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 1.2977971021710195e-07, 'min_child_weight': 20, 'lambda': 99.4385979398534, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
2.5 seconds
[I 2025-05-31 22:05:52,044] Trial 227 finished with value: -596.0321045000001 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 1.0766867782109426e-08, 'min_child_weight': 32, 'lambda': 140.19857433378183, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
2.6 seconds
[I 2025-05-31 22:05:54,706] Trial 228 finished with value: -591.4422605 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 3.686245727621205e-08, 'min_child_weight': 25, 'lambda': 82.70904973770058, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
2.5 seconds
[I 2025-05-31 22:05:57,277] Trial 229 finished with value: -586.1632385 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 3.483411017104337e-07, 'min_child_weight': 37, 'lambda': 122.18411339671609, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
2.2 seconds
[I 2025-05-31 22:05:59,477] Trial 230 finished with value: -584.6388549999999 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 9, 'gamma': 2.425725480886774e-08, 'min_child_weight': 6, 'lambda': 58.03576091356325, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
2.6 seconds
[I 2025-05-31 22:06:02,061] Trial 231 finished with value: -590.565796 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 9, 'gamma': 2.9553420346842775e-08, 'min_child_weight': 37, 'lambda': 124.0858881061765, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
[I 2025-05-31 22:06:03,071] Trial 232 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:06:03,727] Trial 233 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:06:04,313] Trial 234 pruned. Trial was pruned at iteration 20.
2.5 seconds
[I 2025-05-31 22:06:06,798] Trial 235 finished with value: -575.8400575000001 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 9, 'gamma': 1.828715276630296e-08, 'min_child_weight': 47, 'lambda': 105.62314836376376, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
[I 2025-05-31 22:06:07,456] Trial 236 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:06:08,083] Trial 237 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:06:09,049] Trial 238 pruned. Trial was pruned at iteration 20.
2.7 seconds
[I 2025-05-31 22:06:11,733] Trial 239 finished with value: -594.649414 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 1.5972716106758723e-07, 'min_child_weight': 19, 'lambda': 74.7947699790986, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
[I 2025-05-31 22:06:12,356] Trial 240 pruned. Trial was pruned at iteration 20.
2.5 seconds
[I 2025-05-31 22:06:14,905] Trial 241 finished with value: -579.7512815 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 7.970004695507159e-08, 'min_child_weight': 37, 'lambda': 62.56125928054887, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
2.6 seconds
[I 2025-05-31 22:06:17,537] Trial 242 finished with value: -587.650177 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 5.281936028948404e-08, 'min_child_weight': 21, 'lambda': 47.20155076283943, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
2.6 seconds
[I 2025-05-31 22:06:20,167] Trial 243 finished with value: -588.892548 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 2.5187484120465705e-08, 'min_child_weight': 29, 'lambda': 84.81796306896985, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
2.9 seconds
[I 2025-05-31 22:06:23,139] Trial 244 finished with value: -586.701111 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 1.5415379025975205e-07, 'min_child_weight': 9, 'lambda': 91.7623263209023, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
2.7 seconds
[I 2025-05-31 22:06:25,894] Trial 245 finished with value: -582.4947205 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 1.0257265459958567e-07, 'min_child_weight': 20, 'lambda': 34.10167957772191, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
2.6 seconds
[I 2025-05-31 22:06:28,542] Trial 246 finished with value: -594.1955565000001 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 1.0002344225196907e-08, 'min_child_weight': 37, 'lambda': 135.22226431952606, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
2.8 seconds
[I 2025-05-31 22:06:31,344] Trial 247 finished with value: -580.6051635 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 2.315365526973836e-08, 'min_child_weight': 10, 'lambda': 44.118699708844076, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
[I 2025-05-31 22:06:32,042] Trial 248 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:06:32,695] Trial 249 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:06:33,374] Trial 250 pruned. Trial was pruned at iteration 20.
2.5 seconds
[I 2025-05-31 22:06:35,926] Trial 251 finished with value: -567.3463135 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 10, 'gamma': 0.016852493034729323, 'min_child_weight': 1, 'lambda': 101.61440582642247, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
3.0 seconds
[I 2025-05-31 22:06:38,893] Trial 252 finished with value: -577.5386655 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 9, 'gamma': 8.116682373883527e-07, 'min_child_weight': 10, 'lambda': 146.75721962166006, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
[I 2025-05-31 22:06:39,958] Trial 253 pruned. Trial was pruned at iteration 20.
2.3 seconds
[I 2025-05-31 22:06:42,247] Trial 254 finished with value: -590.5600585 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 10, 'gamma': 1.850608772457e-07, 'min_child_weight': 32, 'lambda': 135.608562817217, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
[I 2025-05-31 22:06:42,957] Trial 255 pruned. Trial was pruned at iteration 20.
2.6 seconds
[I 2025-05-31 22:06:45,585] Trial 256 finished with value: -589.964172 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 2.245589535927039e-08, 'min_child_weight': 30, 'lambda': 79.483470922049, 'device': 'cpu'}. Best is trial 187 with value: -599.183777.
2.7 seconds
[I 2025-05-31 22:06:48,309] Trial 257 finished with value: -599.8393245 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 1.1778167700003365e-07, 'min_child_weight': 34, 'lambda': 138.2915405612491, 'device': 'cpu'}. Best is trial 257 with value: -599.8393245.
[I 2025-05-31 22:06:49,003] Trial 258 pruned. Trial was pruned at iteration 20.
2.6 seconds
[I 2025-05-31 22:06:51,647] Trial 259 finished with value: -601.2568665 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 9, 'gamma': 2.484356935538712e-08, 'min_child_weight': 42, 'lambda': 128.45728606916686, 'device': 'cpu'}. Best is trial 259 with value: -601.2568665.
[I 2025-05-31 22:06:52,340] Trial 260 pruned. Trial was pruned at iteration 20.
2.3 seconds
[I 2025-05-31 22:06:54,669] Trial 261 finished with value: -600.418274 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 10, 'gamma': 3.40766898995286e-07, 'min_child_weight': 16, 'lambda': 120.93817589525166, 'device': 'cpu'}. Best is trial 259 with value: -601.2568665.
[I 2025-05-31 22:06:55,339] Trial 262 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:06:55,991] Trial 263 pruned. Trial was pruned at iteration 20.
2.5 seconds
[I 2025-05-31 22:06:58,543] Trial 264 finished with value: -577.715637 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 10, 'gamma': 7.93622747665867e-07, 'min_child_weight': 8, 'lambda': 133.98108160182596, 'device': 'cpu'}. Best is trial 259 with value: -601.2568665.
[I 2025-05-31 22:06:59,165] Trial 265 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:06:59,921] Trial 266 pruned. Trial was pruned at iteration 20.
2.6 seconds
[I 2025-05-31 22:07:02,517] Trial 267 finished with value: -586.0458985 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 1.1779090937338665e-08, 'min_child_weight': 44, 'lambda': 83.90074577077645, 'device': 'cpu'}. Best is trial 259 with value: -601.2568665.
3.0 seconds
[I 2025-05-31 22:07:05,547] Trial 268 finished with value: -578.8237 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 4.685781555966457e-08, 'min_child_weight': 8, 'lambda': 147.47492285154524, 'device': 'cpu'}. Best is trial 259 with value: -601.2568665.
[I 2025-05-31 22:07:06,186] Trial 269 pruned. Trial was pruned at iteration 20.
2.6 seconds
[I 2025-05-31 22:07:08,848] Trial 270 finished with value: -590.4955445 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 9, 'gamma': 1.9992419984399783e-08, 'min_child_weight': 26, 'lambda': 139.1161729496995, 'device': 'cpu'}. Best is trial 259 with value: -601.2568665.
2.6 seconds
[I 2025-05-31 22:07:11,507] Trial 271 finished with value: -583.5397645 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 8, 'gamma': 4.146504974654282e-08, 'min_child_weight': 11, 'lambda': 70.22806166828269, 'device': 'cpu'}. Best is trial 259 with value: -601.2568665.
[I 2025-05-31 22:07:12,140] Trial 272 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:07:12,813] Trial 273 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:07:13,483] Trial 274 pruned. Trial was pruned at iteration 20.
2.7 seconds
[I 2025-05-31 22:07:16,154] Trial 275 finished with value: -595.073761 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 3.05183859418107e-08, 'min_child_weight': 30, 'lambda': 149.38307761644515, 'device': 'cpu'}. Best is trial 259 with value: -601.2568665.
[I 2025-05-31 22:07:16,741] Trial 276 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:07:17,396] Trial 277 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:07:18,041] Trial 278 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:07:18,721] Trial 279 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:07:19,395] Trial 280 pruned. Trial was pruned at iteration 20.
2.7 seconds
[I 2025-05-31 22:07:22,155] Trial 281 finished with value: -598.1211545 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 3.6064478854224275e-08, 'min_child_weight': 27, 'lambda': 148.19527444763497, 'device': 'cpu'}. Best is trial 259 with value: -601.2568665.
[I 2025-05-31 22:07:22,810] Trial 282 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:07:23,526] Trial 283 pruned. Trial was pruned at iteration 20.
1.7 seconds
[I 2025-05-31 22:07:25,271] Trial 284 finished with value: -578.2234195 and parameters: {'stabilization': 'L2', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 9, 'gamma': 2.1804432730529276e-08, 'min_child_weight': 24, 'lambda': 73.82695113896183, 'device': 'cpu'}. Best is trial 259 with value: -601.2568665.
[I 2025-05-31 22:07:26,118] Trial 285 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:07:26,777] Trial 286 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:07:27,499] Trial 287 pruned. Trial was pruned at iteration 20.
2.6 seconds
[I 2025-05-31 22:07:30,163] Trial 288 finished with value: -599.8133545000001 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 1.2879836163321456e-08, 'min_child_weight': 31, 'lambda': 135.28244098710138, 'device': 'cpu'}. Best is trial 259 with value: -601.2568665.
2.6 seconds
[I 2025-05-31 22:07:32,804] Trial 289 finished with value: -580.6707765 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 9, 'gamma': 6.123553663097115e-08, 'min_child_weight': 6, 'lambda': 141.68950138682072, 'device': 'cpu'}. Best is trial 259 with value: -601.2568665.
2.9 seconds
[I 2025-05-31 22:07:35,742] Trial 290 finished with value: -593.4555355 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 9, 'gamma': 1.7887710291476416e-06, 'min_child_weight': 25, 'lambda': 148.21265902499152, 'device': 'cpu'}. Best is trial 259 with value: -601.2568665.
[I 2025-05-31 22:07:36,466] Trial 291 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:07:37,486] Trial 292 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:07:38,254] Trial 293 pruned. Trial was pruned at iteration 20.
2.7 seconds
[I 2025-05-31 22:07:41,016] Trial 294 finished with value: -601.03244 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 8, 'gamma': 9.927857644127e-07, 'min_child_weight': 20, 'lambda': 149.73001916878425, 'device': 'cpu'}. Best is trial 259 with value: -601.2568665.
[I 2025-05-31 22:07:41,767] Trial 295 pruned. Trial was pruned at iteration 20.
3.0 seconds
[I 2025-05-31 22:07:44,814] Trial 296 finished with value: -593.3292845000001 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 9, 'gamma': 5.163741280677277e-07, 'min_child_weight': 21, 'lambda': 143.26665545872834, 'device': 'cpu'}. Best is trial 259 with value: -601.2568665.
3.1 seconds
[I 2025-05-31 22:07:47,920] Trial 297 finished with value: -574.4964904999999 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 8, 'gamma': 1.530174009981062e-05, 'min_child_weight': 8, 'lambda': 82.78390707661082, 'device': 'cpu'}. Best is trial 259 with value: -601.2568665.
2.8 seconds
[I 2025-05-31 22:07:50,760] Trial 298 finished with value: -590.5969545 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 8, 'gamma': 1.4833979383801682e-05, 'min_child_weight': 19, 'lambda': 147.89898730785384, 'device': 'cpu'}. Best is trial 259 with value: -601.2568665.
[I 2025-05-31 22:07:51,566] Trial 299 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:07:52,289] Trial 300 pruned. Trial was pruned at iteration 20.
2.7 seconds
[I 2025-05-31 22:07:55,020] Trial 301 finished with value: -581.532959 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 7, 'gamma': 6.097552892419677e-07, 'min_child_weight': 7, 'lambda': 123.084781251413, 'device': 'cpu'}. Best is trial 259 with value: -601.2568665.
3.0 seconds
[I 2025-05-31 22:07:58,006] Trial 302 finished with value: -577.7270510000001 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 8, 'gamma': 2.9522629235947133e-05, 'min_child_weight': 8, 'lambda': 132.027773687951, 'device': 'cpu'}. Best is trial 259 with value: -601.2568665.
2.9 seconds
[I 2025-05-31 22:08:00,962] Trial 303 finished with value: -595.3218079999999 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 1.1200373778907926e-06, 'min_child_weight': 35, 'lambda': 128.35845793331518, 'device': 'cpu'}. Best is trial 259 with value: -601.2568665.
[I 2025-05-31 22:08:01,674] Trial 304 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:08:02,384] Trial 305 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:08:03,040] Trial 306 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:08:03,686] Trial 307 pruned. Trial was pruned at iteration 20.
3.2 seconds
[I 2025-05-31 22:08:06,957] Trial 308 finished with value: -584.5502625 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 3.103955393656e-07, 'min_child_weight': 9, 'lambda': 140.04559113424372, 'device': 'cpu'}. Best is trial 259 with value: -601.2568665.
[I 2025-05-31 22:08:07,705] Trial 309 pruned. Trial was pruned at iteration 20.
3.2 seconds
[I 2025-05-31 22:08:10,884] Trial 310 finished with value: -590.869873 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 1.3169065943220413e-06, 'min_child_weight': 25, 'lambda': 141.09975241296178, 'device': 'cpu'}. Best is trial 259 with value: -601.2568665.
[I 2025-05-31 22:08:11,593] Trial 311 pruned. Trial was pruned at iteration 20.
3.0 seconds
[I 2025-05-31 22:08:14,582] Trial 312 finished with value: -585.5322575 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 9, 'gamma': 8.48831764817078e-07, 'min_child_weight': 21, 'lambda': 141.69335248673582, 'device': 'cpu'}. Best is trial 259 with value: -601.2568665.
[I 2025-05-31 22:08:15,295] Trial 313 pruned. Trial was pruned at iteration 20.
3.4 seconds
[I 2025-05-31 22:08:18,680] Trial 314 finished with value: -585.3191225 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 2.1299380446707428e-07, 'min_child_weight': 7, 'lambda': 56.399728670769235, 'device': 'cpu'}. Best is trial 259 with value: -601.2568665.
[I 2025-05-31 22:08:19,391] Trial 315 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:08:20,039] Trial 316 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:08:20,675] Trial 317 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:08:21,394] Trial 318 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:08:22,114] Trial 319 pruned. Trial was pruned at iteration 20.
2.7 seconds
[I 2025-05-31 22:08:24,843] Trial 320 finished with value: -596.6705320000001 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 9, 'gamma': 1.979502058044974e-06, 'min_child_weight': 34, 'lambda': 136.30208706641133, 'device': 'cpu'}. Best is trial 259 with value: -601.2568665.
2.9 seconds
[I 2025-05-31 22:08:27,770] Trial 321 finished with value: -590.3934325 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 3.8576995871370396e-06, 'min_child_weight': 21, 'lambda': 146.07312706661165, 'device': 'cpu'}. Best is trial 259 with value: -601.2568665.
2.8 seconds
[I 2025-05-31 22:08:30,572] Trial 322 finished with value: -601.0820925 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 8, 'gamma': 8.482319002406128e-07, 'min_child_weight': 26, 'lambda': 137.64263963352425, 'device': 'cpu'}. Best is trial 259 with value: -601.2568665.
3.3 seconds
[I 2025-05-31 22:08:33,927] Trial 323 finished with value: -572.7724304999999 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 9, 'gamma': 2.140104287538964e-08, 'min_child_weight': 2, 'lambda': 130.57280610745346, 'device': 'cpu'}. Best is trial 259 with value: -601.2568665.
2.5 seconds
[I 2025-05-31 22:08:36,440] Trial 324 finished with value: -584.5784914999999 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 8, 'gamma': 8.703545718421913e-06, 'min_child_weight': 51, 'lambda': 74.90293344038497, 'device': 'cpu'}. Best is trial 259 with value: -601.2568665.
[I 2025-05-31 22:08:37,183] Trial 325 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:08:37,865] Trial 326 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:08:38,564] Trial 327 pruned. Trial was pruned at iteration 20.
2.8 seconds
[I 2025-05-31 22:08:41,416] Trial 328 finished with value: -580.2828675000001 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 9, 'gamma': 6.112702558674326e-07, 'min_child_weight': 12, 'lambda': 119.17254387580842, 'device': 'cpu'}. Best is trial 259 with value: -601.2568665.
2.1 seconds
[I 2025-05-31 22:08:43,546] Trial 329 finished with value: -592.4882505 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 6, 'gamma': 2.1310682877225877e-05, 'min_child_weight': 11, 'lambda': 44.371533861121705, 'device': 'cpu'}. Best is trial 259 with value: -601.2568665.
2.6 seconds
[I 2025-05-31 22:08:46,182] Trial 330 finished with value: -604.271698 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 8, 'gamma': 1.1232379176730886e-07, 'min_child_weight': 39, 'lambda': 137.23900983865695, 'device': 'cpu'}. Best is trial 330 with value: -604.271698.
2.0 seconds
[I 2025-05-31 22:08:48,254] Trial 331 finished with value: -598.9309695 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 5, 'gamma': 3.695886342690696e-08, 'min_child_weight': 21, 'lambda': 85.06102079346542, 'device': 'cpu'}. Best is trial 330 with value: -604.271698.
[I 2025-05-31 22:08:48,982] Trial 332 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:08:49,643] Trial 333 pruned. Trial was pruned at iteration 20.
2.8 seconds
[I 2025-05-31 22:08:52,494] Trial 334 finished with value: -582.6651305 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 8, 'gamma': 7.1431361852683e-07, 'min_child_weight': 12, 'lambda': 122.0100787047589, 'device': 'cpu'}. Best is trial 330 with value: -604.271698.
1.9 seconds
[I 2025-05-31 22:08:54,454] Trial 335 finished with value: -587.1540225 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 4, 'gamma': 5.551209154082488e-08, 'min_child_weight': 35, 'lambda': 67.56774173764688, 'device': 'cpu'}. Best is trial 330 with value: -604.271698.
[I 2025-05-31 22:08:55,189] Trial 336 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:08:55,861] Trial 337 pruned. Trial was pruned at iteration 20.
3.0 seconds
[I 2025-05-31 22:08:58,920] Trial 338 finished with value: -595.250183 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 8, 'gamma': 4.4973334398682754e-08, 'min_child_weight': 6, 'lambda': 144.17501991657653, 'device': 'cpu'}. Best is trial 330 with value: -604.271698.
2.1 seconds
[I 2025-05-31 22:09:01,088] Trial 339 finished with value: -590.0383605 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 6, 'gamma': 6.730954119158129e-08, 'min_child_weight': 30, 'lambda': 77.32880141433776, 'device': 'cpu'}. Best is trial 330 with value: -604.271698.
[I 2025-05-31 22:09:01,829] Trial 340 pruned. Trial was pruned at iteration 20.
2.2 seconds
[I 2025-05-31 22:09:04,025] Trial 341 finished with value: -585.855011 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 6, 'gamma': 2.071911108729702e-08, 'min_child_weight': 7, 'lambda': 102.50620923720858, 'device': 'cpu'}. Best is trial 330 with value: -604.271698.
[I 2025-05-31 22:09:04,772] Trial 342 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:09:05,394] Trial 343 pruned. Trial was pruned at iteration 20.
2.7 seconds
[I 2025-05-31 22:09:08,121] Trial 344 finished with value: -596.4279175 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 8, 'gamma': 3.195613146365003e-08, 'min_child_weight': 22, 'lambda': 125.95327754417856, 'device': 'cpu'}. Best is trial 330 with value: -604.271698.
[I 2025-05-31 22:09:08,843] Trial 345 pruned. Trial was pruned at iteration 20.
1.9 seconds
[I 2025-05-31 22:09:10,759] Trial 346 finished with value: -571.1947325 and parameters: {'stabilization': 'L2', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 7, 'gamma': 1.874713140922482e-07, 'min_child_weight': 15, 'lambda': 141.35966042376964, 'device': 'cpu'}. Best is trial 330 with value: -604.271698.
[I 2025-05-31 22:09:11,508] Trial 347 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:09:12,184] Trial 348 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:09:12,855] Trial 349 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:09:13,535] Trial 350 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:09:14,250] Trial 351 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:09:14,889] Trial 352 pruned. Trial was pruned at iteration 20.
2.7 seconds
[I 2025-05-31 22:09:17,610] Trial 353 finished with value: -600.6770325 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 8, 'gamma': 3.905926756821402e-08, 'min_child_weight': 19, 'lambda': 84.46157687717987, 'device': 'cpu'}. Best is trial 330 with value: -604.271698.
[I 2025-05-31 22:09:18,329] Trial 354 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:09:19,011] Trial 355 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:09:19,723] Trial 356 pruned. Trial was pruned at iteration 20.
2.9 seconds
[I 2025-05-31 22:09:22,672] Trial 357 finished with value: -585.9804690000001 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 8, 'gamma': 1.9445567469897204e-08, 'min_child_weight': 9, 'lambda': 148.50848226274172, 'device': 'cpu'}. Best is trial 330 with value: -604.271698.
[I 2025-05-31 22:09:23,407] Trial 358 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:09:24,013] Trial 359 pruned. Trial was pruned at iteration 20.
2.5 seconds
[I 2025-05-31 22:09:26,515] Trial 360 finished with value: -589.624054 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 8, 'gamma': 8.006094627689205e-08, 'min_child_weight': 43, 'lambda': 119.59372058859712, 'device': 'cpu'}. Best is trial 330 with value: -604.271698.
[I 2025-05-31 22:09:27,341] Trial 361 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:09:28,033] Trial 362 pruned. Trial was pruned at iteration 20.
2.0 seconds
[I 2025-05-31 22:09:30,031] Trial 363 finished with value: -586.2982179999999 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'approx', 'max_depth': 4, 'gamma': 2.046189509021907e-07, 'min_child_weight': 21, 'lambda': 114.07050633904194, 'device': 'cpu'}. Best is trial 330 with value: -604.271698.
[I 2025-05-31 22:09:30,706] Trial 364 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:09:31,357] Trial 365 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:09:32,123] Trial 366 pruned. Trial was pruned at iteration 21.
[I 2025-05-31 22:09:32,779] Trial 367 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:09:33,515] Trial 368 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:09:34,214] Trial 369 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:09:34,842] Trial 370 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:09:35,465] Trial 371 pruned. Trial was pruned at iteration 20.
[I 2025-05-31 22:09:36,214] Trial 372 pruned. Trial was pruned at iteration 20.
2.7 seconds
[I 2025-05-31 22:09:38,941] Trial 373 finished with value: -574.2764895 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 7, 'gamma': 3.127980740432347e-08, 'min_child_weight': 10, 'lambda': 116.8959020481343, 'device': 'cpu'}. Best is trial 330 with value: -604.271698.
2.6 seconds
[I 2025-05-31 22:09:41,562] Trial 374 finished with value: -581.2537845 and parameters: {'stabilization': 'None', 'response_fn': 'exp', 'eta': 0.5, 'tree_method': 'hist', 'max_depth': 10, 'gamma': 0.0063497907083119925, 'min_child_weight': 51, 'lambda': 80.59123209789261, 'device': 'cpu'}. Best is trial 330 with value: -604.271698.

Hyper-Parameter Optimization successfully finished.
Number of finished trials:  375
	Best trial:
		Value: -604.271698
		Params:
			stabilization: None
			response_fn: exp
			eta: 0.5
			tree_method: hist
			max_depth: 8
			gamma: 1.1232379176730886e-07
			min_child_weight: 39
			lambda: 137.23900983865695
			device: cpu
			opt_rounds: 39

Now, with this optimized set of parameters, we can start training models! First, let's reduce the training rate (eta) to a lower value so that we can find the optimal number of boosting rounds. We will train a model with the training set and evaluate it with the validation set at each iteration. The iteration set after which the loss on the valdation set begins to increase will be the optimal number of rounds (this is known as early stopping, see the docs here for more information).

In [100]:

# Set learning rate in best params to a smaller value
best_params_1stpass['eta'] = 0.01

# Alternatively, you can overwrite the eta saved in the tuning params in the AGNBoost instance
# Then, you can run agnboost_m.train_model without passing in a params dict, as it will by default use the tuned params.
#  agnboost_m.model_info['agn.fracAGN']['tuned_params']['eta'] = 0.01

model, eval_results = agnboost_m.train_model( model_name = 'agn.fracAGN',
                                           dtrain = catalog, 
                                             params = best_params_1stpass,
                                           split_type='train', # Use the training split for training
                                            dval = True,  # Evaluate with the validation set at each training step.
                                            early_stopping_rounds = 50, # When the loss on the vlidation set increases after 50 consecutive iterations, stop training.
                                            num_boost_round = 10**4 # The maximm number of boosting rounds
                                          )

# Print the best iteration
opt_rounds = agnboost_m.model_info['agn.fracAGN']['best_iteration']
print(f"Optimal number of boosting rounds: {opt_rounds}")

# Plot the trainign and validation loss curves
ax = agnboost_m.plot_eval( evals = eval_results, catalog = catalog, best_iter = opt_rounds )

# Set learning rate in best params to a smaller value best_params_1stpass['eta'] = 0.01 # Alternatively, you can overwrite the eta saved in the tuning params in the AGNBoost instance # Then, you can run agnboost_m.train_model without passing in a params dict, as it will by default use the tuned params. # agnboost_m.model_info['agn.fracAGN']['tuned_params']['eta'] = 0.01 model, eval_results = agnboost_m.train_model( model_name = 'agn.fracAGN', dtrain = catalog, params = best_params_1stpass, split_type='train', # Use the training split for training dval = True, # Evaluate with the validation set at each training step. early_stopping_rounds = 50, # When the loss on the vlidation set increases after 50 consecutive iterations, stop training. num_boost_round = 10**4 # The maximm number of boosting rounds ) # Print the best iteration opt_rounds = agnboost_m.model_info['agn.fracAGN']['best_iteration'] print(f"Optimal number of boosting rounds: {opt_rounds}") # Plot the trainign and validation loss curves ax = agnboost_m.plot_eval( evals = eval_results, catalog = catalog, best_iter = opt_rounds )

2025-06-01 00:36:07,607 - AGNBoost.AGNBoost - WARNING - Catalog object passsed. Taking the features and labels of the train set stored in the passed Catalog.

16.9 seconds to train model
Optimal number of boosting rounds: 2050

In [ ]: