Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
864 views
in Technique[技术] by (71.8m points)

python - f1_score metric in lightgbm

I want to train a lgb model with custom metric : f1_score with weighted average.

I went through the advanced examples of lightgbm over here and found the implimentation of custom binary error function. I implemented as similiar functon to return f1_score as shown below.

def f1_metric(preds, train_data):

    labels = train_data.get_label()

    return 'f1', f1_score(labels, preds, average='weighted'), True

I tried to train the model by passing feval parameter as f1_metric as shown below.

evals_results = {}

bst = lgb.train(params, 
                     dtrain, 
                     valid_sets= [dvalid], 
                     valid_names=['valid'], 
                     evals_result=evals_results, 
                     num_boost_round=num_boost_round,
                     early_stopping_rounds=early_stopping_rounds,
                     verbose_eval=25, 
                     feval=f1_metric)

Then I am getting ValueError: Found input variables with inconsistent numbers of samples:

The training set is being passed to the function rather than the validation set.

How can I configure such that the validation set is passed and f1_score is returned.?

See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

The docs are a bit confusing. When describing the signature of the function that you pass to feval, they call its parameters preds and train_data, which is a bit misleading.

But the following seems to work:

from sklearn.metrics import f1_score

def lgb_f1_score(y_hat, data):
    y_true = data.get_label()
    y_hat = np.round(y_hat) # scikits f1 doesn't like probabilities
    return 'f1', f1_score(y_true, y_hat), True

evals_result = {}

clf = lgb.train(param, train_data, valid_sets=[val_data, train_data], valid_names=['val', 'train'], feval=lgb_f1_score, evals_result=evals_result)

lgb.plot_metric(evals_result, metric='f1')

To use more than one custom metric, define one overall custom metrics function just like above, in which you calculate all metrics and return a list of tuples.

Edit: Fixed code, of course with F1 bigger is better should be set to True.


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...