Optuna

설치 방법 : pip install optuna ( Python 3.6 이상 지원된다.)

https://optuna.org/

import optuna

def objective(trial):
    x = trial.suggest_uniform('x', -10, 10)
    return (x - 2) ** 2

study = optuna.create_study()
study.optimize(objective, n_trials=100)

study.best_params  # E.g. {'x': 2.002108042}
'''
[I 2021-05-22 21:50:25,855] Trial 0 finished with value: 4.901344506787557 and parameters: {'x': -0.21389803441521593}. Best is trial 0 with value: 4.901344506787557.
[I 2021-05-22 21:50:25,856] Trial 1 finished with value: 19.383686822446574 and parameters: {'x': 6.402690861558028}. Best is trial 0 with value: 4.901344506787557.
[I 2021-05-22 21:50:25,857] Trial 2 finished with value: 48.26980998276973 and parameters: {'x': -4.947647802153599}. Best is trial 0 with value: 4.901344506787557.
[I 2021-05-22 21:50:25,857] Trial 3 finished with value: 2.2924231332863716 and parameters: {'x': 3.514075009134743}. Best is trial 3 with value: 2.2924231332863716.
[I 2021-05-22 21:50:25,858] Trial 4 finished with value: 50.07509975290213 and parameters: {'x': 9.0763761737843}. Best is trial 3 with value: 2.2924231332863716.
[I 2021-05-22 21:50:25,858] Trial 5 finished with value: 36.42002000004158 and parameters: {'x': -4.034900164877757}. Best is trial 3 with value: 2.2924231332863716.
[I 2021-05-22 21:50:25,859] Trial 6 finished with value: 43.68813229659039 and parameters: {'x': 8.60969986433502}. Best 
is trial 3 with value: 2.2924231332863716.
'''(중략)'''
[I 2021-05-22 21:50:26,067] Trial 95 finished with value: 1.0808769981917132 and parameters: {'x': 0.9603476551309504}. Best is trial 22 with value: 0.0001617458402431121.
[I 2021-05-22 21:50:26,069] Trial 96 finished with value: 0.8810874379171446 and parameters: {'x': 2.93866257937405}. Best is trial 22 with value: 0.0001617458402431121.
[I 2021-05-22 21:50:26,072] Trial 97 finished with value: 5.522709236341806 and parameters: {'x': -0.3500445179489273}. Best is trial 22 with value: 0.0001617458402431121.
[I 2021-05-22 21:50:26,075] Trial 98 finished with value: 0.3600695597435451 and parameters: {'x': 1.3999420363468666}. Best is trial 22 with value: 0.0001617458402431121.
[I 2021-05-22 21:50:26,077] Trial 99 finished with value: 0.003693533347135864 and parameters: {'x': 2.0607744464979803}. Best is trial 22 with value: 0.0001617458402431121.
'''

def objective(trial):
    from sklearn.svm import SVC
    params = {
        'C': trial.suggest_loguniform('C', 0.01, 0.1),
        'gamma': trial.suggest_categorical('gamma', ["auto"]),
        'kernel': trial.suggest_categorical("kernel", ["rbf"])
    }

    svc = SVC(**params, verbose=True)
    svc.fit(X_train, y_train)
    return svc.score(X_test, y_test)

study = optuna.create_study(sampler=optuna.samplers.TPESampler(seed=123),
                            direction="maximize",
                            pruner=optuna.pruners.MedianPruner())
study.optimize(objective, n_trials=5, show_progress_bar=True)

print(f"Best Value from optune: {study.best_trial.value}")
print(f"Best Params from optune: {study.best_params}")

 

 (캐글의 타이타닉 대회에서) SVC에 적용해 본 코드다. 아래처럼 나온다. 

Best Value from optune: 0.875575
Best Params from optune: {'C': 0.052416614815162584, 'gamma': 'auto', 'kernel': 'rbf'}