Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
916 views
in Technique[技术] by (71.8m points)

python 3.x - scikit-lean GridSearchCV n_jobs != 1 freezing

I'm running grid search on random forests and trying to use n_jobs different than one but the kernel freezes, there is no CPU usage. With n_jobs=1 it works fine. I can't even stop the command with ctl-C and have to restart the kernel. I'm running on windows 7. I saw that there is a similar problem with OS X but the solution is not relevant for windows 7.

from sklearn.ensemble import RandomForestClassifier
rf_tfdidf = Pipeline([('vect',tfidf),
                  ('clf', RandomForestClassifier(n_estimators=50, 
class_weight='balanced_subsample'))])

param_grid = [{'vect__ngram_range':[(1,1)],
          'vect__stop_words': [stop],
          'vect__tokenizer':[tokenizer]
          }]
if __name__ == '__main__':
gs_rf_tfidf = GridSearchCV(rf_tfdidf, param_grid, scoring='accuracy', cv=5, 
                                                           verbose=10, 
                                                           n_jobs=2)
gs_rf_tfidf.fit(X_train_part, y_train_part)

thanks.

See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

The indent after if __name__ == '__main__': is not correct. If it's not the case and it's a copy paste mistake then you can try something like :

if __name__ =='main':
    # your code indented !

So the first line of your script is if __name__ == '__main__': and then the rest code follows with the appropriate indent.

New Code

from sklearn.ensemble import RandomForestClassifier
from sklearn.pipeline import Pipeline    

if __name__ == '__main__':

    rf_tfdidf = Pipeline([('vect',tfidf),('clf', RandomForestClassifier(n_estimators=50,class_weight='balanced_subsample'))])

    param_grid = [{'vect__ngram_range':[(1,1)],'vect__stop_words': [stop],'vect__tokenizer':[tokenizer]}]

    gs_rf_tfidf = GridSearchCV(rf_tfdidf, param_grid, scoring='accuracy', cv=5,verbose=10, n_jobs=-1)

    gs_rf_tfidf.fit(X_train_part, y_train_part)

This works fine for me (windows 8.1)

EDIT

The following works fine using PyCharm. I have not used spyder but it should also work for spyder:

Code

Class Test(object):
    def __init__(self):
        ###code here
        ###code here    

if __name__ == '__main__':
    Test()

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...