Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
4.3k views
in Technique[技术] by (71.8m points)

precision recall - How do I define a custom eval metric in Catboost (python) compatible with class_weights?

Using this example, I created a precision-recall AUC eval metric for Catboost. However, I need some guidance on how to make it compatible with the class_weights argument in which I will be passing a list (example: [625.0, 0.500400320256205]) because I have a large class imbalance. Below is my custom eval metric:

from sklearn.metrics import average_precision_score
import numpy as np
from scipy.special import expit

# define pr-AUC custom eval metric
class PrecisionRecallAUC:
    # define a static method to use in evaluate method
    @staticmethod
    def get_pr_auc(y_true, y_pred):
        # fit predictions to logistic sigmoid function
        y_pred = expit(y_pred).astype(float)
        # actual values should be 1 or 0 integers
        y_true = y_true.astype(int)
        # calculate average precision
        flt_pr_auc = average_precision_score(y_true=y_true, 
                                             y_score=y_pred)
        
        return flt_pr_auc
    
    # define a function to tell catboost that greater is better (or not)
    def is_max_optimal(self):
        # greater is better
        return True

    # get the score
    def evaluate(self, approxes, target, weight):
        # make sure length of approxes == 1
        assert len(approxes) == 1
        # make sure length of target is the same as predictions
        assert len(target) == len(approxes[0])
        # set target to integer and save as y_true
        y_true = np.array(target).astype(int)
        # save predictions
        y_pred = approxes[0]
        # generate score
        score = self.get_pr_auc(y_true=y_true, y_pred=y_pred)
        return score, 1

    # return score
    def get_final_error(self, error, weight):
        return error

How do I use my list of class weights in this custom eval metric?

Thank you in advance.


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)
等待大神答复

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...