我是如何准备机器学习竞赛，超详细的综合指南-FlyAI

我是如何准备机器学习竞赛，超详细的综合指南

机器学习深度学习

最后更新 2020-12-01 15:15 阅读 5775

最后更新 2020-12-01 15:15

阅读 5775

机器学习深度学习

##FlyAI文献翻译
**英文原文：**[：Comprehensive Guide for Machine Learning Competitions](https://vinscoder1627.medium.com/comprehensive-guide-for-machine-learning-competitions-7ae71fb1700e)

> **标签** ：强化学习 机器学习

![](https://static.leiphone.com/uploads/new/sns/article/202011/1605507269625791.png)

Machine Learning is easy !

>  机器学习很简单

This article is meant to serve as a template for beginners and experts alike for getting started with Machine Learning Competitions.

Learning Machine Learning theoretically, understanding all those awesome algorithms is fun but the most important part is to apply those concepts practically.

Doing things practically enhances your concepts and helps you unveil significant findings that can’t be achieved via theory alone.

The best way to hone your skills is to take part in real world competitions where you can find lots of like minded people competing together to find solutions for the given problem statement. It’s not always important to win the competition but to recognize the factors and pipelines undertaken by other people to achieve outstanding results.

>本文旨在为初学者和专家提供一个模板，让他们能够开始学习机器学习竞赛。

>从理论上学习机器学习，理解所有这些令人惊叹的算法是很意思的，但最重要的部分是实际应用这些概念。

>实践活动可以加强你对概念的理解，并会向你展示仅通过理论无法达到的重大发现。

>增强技能的最佳途径是参加现实世界的比赛，在那里你可以找到很多志同道合的人一起竞争，为给定的问题描述找到解决方案。赢得比赛并不总是重要的，而是要认识到其他人为了取得出色的结果所采取的方法和流程。

![](https://static.leiphone.com/uploads/new/sns/article/202011/1605507266456572.png)

Everyone is a Winner !

A few websites where you can experiment with all the ideas you have !

>在一些网站上，你可以用实验验证自己所有的想法!

[Kaggle](https://www.kaggle.com/ "Kaggle")

[Driven Data](https://www.drivendata.org/ "Driven Data")

[Dphi](https://dphi.tech/practice/ "Dphi")

[Machine Hack](https://www.machinehack.com/ "Machine Hack")

To find a list of all the ongoing Competitions refer to this [link](https://mlcontests.com/ "link").

>要查找所有正在进行的比赛列表，请参考这个链接。

There are 2 Bonus sections included to bring out the best results. So make sure you read till the end.

>包括2个“奖金”部分，包括如何得到最好的结果。因此，请确保您阅读到最后。

##Getting Started !

One of the most important thing in Machine Learning is to understand the data. Most of the online competitions are based on supervised Machine Learning where some kind of prediction needs to be done.

Now this prediction could be a :

1) Regression problem where the output label is a continuous range of integers.

Examples include “predicting the age of the person”, “predicting the price of an item”, “census income”, “compressive strength of materials” etc.

2) Classification problem where the output label is discrete and finite in nature.

Examples include “predicting the gender of the person”, “predicting the success of a startup”, “object recognition”, “text classification”, “sentiment analysis”, “cat vs dog classification” etc.

Still getting confused, let’s take a simple example.

If you are asked to predict the prices of stocks, take a look at the output column, if there are a wide range of values like 6.2, 7.1, 8.8, 9,10 … and predicting “6.5" (something not mentioned in the output column of the training set) is also a possibility, you could safely say that this is a classic regression based problem.

If you are asked to predict the direction of stock and the output label contains +1, 0 and -1 as the sole outputs where “+1” denotes upward rise, “ -1” denotes downward fall and “0" denotes no change. Here, we can clearly notice there are only 3 possibilities for the output label and predicting “6.5” as the output makes no sense. Hence, this becomes a classification based problem wherein each output label infers a special meaning.

Once you have decided the type of algorithms you could use for the given problem, things become systematic and easy.

>##开始吧！

>机器学习中最重要的事就是理解数据。大部分的在线比赛都是有监督机器学习，其中我们需要做一些预测。

>现在预测可能会是：

>1）回归问题，其中输出标签是连续的整数范围。

>例如 "预测人的年龄"、"预测物品的价格"、"人口普查收入"、"材料的抗压强度 "等。

>2）分类问题，输出的标签本质上是离散的且有限的。

>例如包括“预测人的性别”，“预测创业成功与否”，“目标识别”，“文本分类”，“情感分析”，“猫与狗的分类”等。

>或许你仍在疑惑，我们举个简单的例子。

>如果要求你去预测股票价格，看一看输出列，如果有很宽的数值范围，例如6.2、7.1、8.8、9,10…并预测为“ 6.5”（输出中未提及）训练集”一列）也是有可能的，您可以肯定地说这是一个典型的回归问题。 如果要求你预测股票的趋势，并且输出标签包含+1、0和-1作为唯一输出，其中“ +1”表示上升，“-1”表示下降，“ 0”表示不变。在这里，我们可以清楚地注意到输出标签只有3种可能性，预测 "6.5 "作为输出是没有意义的。因此，这就变成了一个基于分类的问题，其中每个输出标签都具有特殊的含义。

>一旦确定了可以用于给定问题的算法的类型，事情就会变得系统化和简单化。

##Loading the Data

In this article we would be considering structured data (highly organized data in excel sheets, csv files etc.). For dealing with Unstructured data (audio, text, images, video files), Deep Learning models take an edge. I will be covering in the upcoming articles.

Pandas is the most commonly used library to load “csv files” and “excel sheets”, as pandas provides lots of powerful data manipulations and aggregations.

Link to the dataset. [[view]](https://drive.google.com/file/d/1sjUNk5YBTcFN90HaflJHmGxcZ-mfK7oW/view?usp=sharing "[view]")

>##载入数据

>在本文中，我们将考虑结构化数据（excel表、csv文件等高度组织化的数据）。对于处理非结构化数据（音频，文本，图像，视频文件），深度学习模型占据优势。我将在接下来的文章中介绍。

>Pandas是加载 "csv文件 "和 "excel表 "最常用的库，因为pandas提供了很多强大的数据处理和聚合功能。

>链接到数据集。[view]

```
import pandas as pd    
import numpy as np    
import matplotlib.pyplot as plt    
# load the data    
data = pd.read_csv("training_data.csv")    
# print the number of rows and columns    
print(data.shape)    
# view the first 5 rows of data    
data.head()
```

With a single line of code, you could read the data present in the file “training_data.csv”

With the help of “data.head()” you could have a look at the rows and columns of the data to estimate the type of Machine Learning algorithm to be applied as discussed earlier [regression or classification].

>只需一行代码，你就可以读取 "training_data.csv "文件中的数据。

>在 "data.head() "的帮助下，你可以查看数据的行和列，以估计要应用的机器学习算法的类型，就像前面讨论过的[回归或分类]。

##Exploratory Data Analysis

Once the data is loaded, the next step is to explore the data and check for irregularities if present.

>##探索式数据分析

>数据加载后，下一步就是探索数据并检查数据是否存在异常。

```
# check for null/ missing values    
data.isna().sum()
```

![](https://static.leiphone.com/uploads/new/sns/article/202011/1605507267456873.png)

Most of the times, we could observe some missing values in each column of our dataset.

By executing the above line, if we see 0 corresponding to all the columns that means there are no missing values.

If you observe some null values in any column, we need to take care of those particular inputs as the model won’t be able to make sense out of null values.

>大多数时候，我们可以观察到数据集的每一列都有一些缺失值。

>通过执行上面的一行代码，如果我们看到所有列对应的0，意味着没有缺失值。

>如果你在任何一列中观察到一些空值，那么我们需要处理这些特定的输入，因为模型将无法理解空值。

It’s just numerical values that the Machine understands !

>这只是机器能理解的数值！

There is a huge difference between “0” and “None”, “0” is the value of that particular feature while “None” implies absence of a value.

It’s possible that during the data preparation, all the values in the csv files are not filled, some might be left blank. The cells which are left blank takes up “NaN”(Not a Number).

Whenever we encounter NaN / missing values, here are a few ways to deal with it.

>"0 "和 "None "之间有很大的区别，"0 "是该特定特征的值，而 "None "则意味着没有值。

>有可能在数据的准备过程中，csv文件中的所有值都没有被填满，有些可能是空白的。留有空白的单元格就占了 "NaN"（Not a Number）。

>每当我们遇到NaN/缺失值时，这里有几种处理方法。

If the number of such rows are very less compared to the total rows in the dataset, you could safely drop such rows.

>如果此类行的数量与数据集中的总行相比要少得多，则可以放心地删除此类行。

```
#Drop the rows where at least one element is missing    
data.dropna()
```

For example: if there are 25 rows have missing values out of 5000 rows in the data, then dropping these rows won’t make much of a difference.

Impute the “NaN” values with some specific values.

>例如：如果数据中有5000行中有25行缺少值，那么删除这些行不会有太大的区别。

>用一些特定值估算“ NaN”值。

```
from sklearn.impute import SimpleImputer    
import numpy as np    
# create an object and specify the required configuration    
# imputing numerical feature values    
imp_1 = SimpleImputer(missing_values= np.nan, strategy="mean")    
numerical_cols = list(data.select_dtypes(include = 'number').columns)    
data[numerical_cols] = imp_1.fit_transform(data[numerical_cols])    
#imputing categorical feature values    
imp_2 = SimpleImputer(missing_values = np.nan, strategy="most_frequent")    
categorical_cols = list(data.select_dtypes(include = 'object').columns)    
data[categorical_cols] = imp_2.fit_transform(data[categorical_cols])
```
We could make use of the SimpleImputer class present in the sklearn library, one of the most famous library for Machine Learning implementations.

Since, we are dealing with missing values ie. “NaN”, so in the first parameter we will be mentioning np.nan.

For strategy, we have 4 options : “mean”, “median”, “most_frequent”, and “constant”.

>我们可以利用sklearn库中存在的SimpleImputer类，它是最著名的机器学习实现库之一。

>由于我们处理的是缺失值，即 "NaN"，所以在第一个参数中，我们将提到np.nan。

>对于策略，我们有4个选项 ："平均值"，"中位数"，"最频繁 "和 "常数"。

Note: “mean” and “median” are statistical measures and therefore could be used only with numerical data. “most_frequent” and “constant” could be used with both numerical as well as categorical data.

After this step, we would be having clean data with no null values.

>注："平均值 "和 "中位数 "是统计量，因此只能用于数值数据。"最频繁 "和 "常数 "既可用于数值数据，也可用于分类数据。

>完成此步骤后，我们将获得没有空值的干净数据。

##Tackling Numerical Features

>##处理数据特征

![](https://static.leiphone.com/uploads/new/sns/article/202011/1605507269735275.png)

We could observe that the max value of the feature “Age” is 60 whereas the max value of “DailyRate” is 1499. We can clearly notice a huge difference in the range of values.

Machine Learning involves elementary addition and multiplication operations, basically there is a weightage corresponding to each feature. Due to obvious reasons, a feature smaller range of values will be assigned more weightage to take its affect into account.

This would lead to a model which is not interpretable in terms of feature importance and ranking.

Do not scale the Output Feature, doing so will lead to disastrous results. Only the input features need to be scaled.

A simple solution would be to scale the values so that all the numerical values span a similar range of values.

>我们可以观察到，特征 "年龄 "的最大值是60，而 " 日常频率"的最大值是1499。我们可以清楚地发现，数值范围有很大的差异。

>机器学习涉及到基本的加法和乘法运算，基本上每个特征都有一个对应的权重。由于显而易见的原因，一个特征较小的数值范围会被赋予更多的权重，以考虑其影响。
>
这将导致模型在特征重要性和排序方面无法解释。

>不要对输出特征进行缩放，这样做会导致灾难性的结果。只需要对输入特征进行缩放。

>一个简单的解决方法是将数值按比例放大，使所有的数值都有类似的数值范围。

```
# lets scale the values.    
from sklearn.preprocessing import MinMaxScaler    
scaler = MinMaxScaler()    
data[numerical_cols] = scaler.fit_transform(data[numerical_cols])
```
![](https://static.leiphone.com/uploads/new/sns/article/202011/1605507269287201.png)

Using the MinMaxScaler class from sklearn library we have scaled all the numerical columns to range from [0, 1].

>使用sklearn库中的MinMaxScaler类，我们已经将所有数值列的范围调整到了[0, 1]。

![](https://static.leiphone.com/uploads/new/sns/article/202011/1605507270260837.png)

##Tackling Categorical Features

>##处理类别特征

```
for i in cat_cols:    
print(i, " : ", data[i].unique())
```
![](https://static.leiphone.com/uploads/new/sns/article/202011/1605507270413666.png)

We could observe the unique values corresponding to each categorical feature. Based on the number of values and the ordinal nature of the values certain methods could be put into action.

Machine Learning models cannot interpret string values, so we need to encode these categorical features into numerical values.

>我们可以观察到每个分类特征对应的唯一值。根据数值的数量和数值的顺序性质，可以应用到某些方法。

>由于机器学习模型无法解释字符串值，所以我们需要将这些分类特征编码成数值。

```
# gender : Female : 0, Male : 1    
data['Gender'] = data['Gender'].map({'Female' : 1, 'Male' : 0})    
data.head()
```

You can make use of pandas map function to manually encode the features.

>你可以利用pandas map功能对特征进行手动编码。

```
from sklearn.preprocessing import LabelEncoder    
le = LabelEncoder()    
data[category_cols] = data[category_cols].apply(le.fit_transform)    
data.head()
```
The simplest way to deal with this problem is to use LabelEncoder. It will start assigning integer values starting from 0 to each new feature label.

>处理这个问题最简单的方法是使用LabelEncoder。它会将从0开始的整数值分配给每个新的特征标签。

Alternatively, you can also one hot encode the values, when you feel all the particular values should receive equal weightage and no special preference is given to a particular category of values.

If you are getting confused, I would suggest to move forward with LabelEncoder. In case, the output feature is also in the form of string values, then you need to encode the output feature / target column as well.

>另外，你也可以对数值进行one-hot编码，当你觉得所有的特定数值都应该得到同等的权重，而不对某一类数值有特殊的偏好。

>如果你还不是很明白，那我建议你继续使用LabelEncoder。如果输出特征也是字符串值的形式，那么你需要对输出特征/目标列也进行编码。

##Well done, your data is now ready to be fed into various Machine Learning Models.

>###好了，你的数据现在已经可以输入到各种机器学习模型中

![](https://static.leiphone.com/uploads/new/sns/article/202011/1605507270459828.png)

Before: missing values, no scaling and encoding

>之前：缺失值，无归一化和编码

![](https://static.leiphone.com/uploads/new/sns/article/202011/1605507270346970.png)

After: Looks systematic !

>之后：看起来更有系统性！

##Getting Data Ready for Model fitting

>##让数据为模型拟合做好准备

```
#Define X and y    
X = data.drop('Attrition', axis = 1)    
y = data['Attrition']    
print("X shape : ", X.shape) # X shape :  (1270, 52)    
print("y shape : ", y.shape) # y shape :  (1270,)
```
‘X’ would be the input features that would be fed to the model during training phase. ‘y’ is the corresponding output labels.

Our aim is given a new set of input feature values, our model should predict the most accurate value for the target feature.

Here, we could observe that there are 52 input features that would be fed to the model.

>'X'将是在训练阶段喂给模型的输入特征。'y'是相应的输出标签。

>我们的目的是给定一组新的输入特征值，我们的模型应该预测目标特征的精确值。

>在这里，我们可以观察到，有52个输入特征将被输入到模型中。

```
from sklearn.model_selection import train_test_split    
# Split the data (80/20)    
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state=42)    
print("X train shape : ", X_train.shape) # X train shape :  (762, 52)    
print("y train shape : ", y_train.shape) # y train shape :  (762,)    
print("X test shape : ", X_test.shape) # X test shape :  (508, 52)    
print("y test shape : ", y_test.shape) # y test shape :  (508,)
```
After defining ‘X’ and ‘y’, we need to split our data set into 2 parts ie. training and testing set.

The model is trained on the training set and to test the model’s performance, we will evaluate it on the testing set(unseen by the model till now).

>在定义了 "X "和 "y "之后，我们需要将数据集分成两部分，即训练集和测试集。

>模型在训练集上进行训练，为了测试模型的性能，我们将在测试集上对其进行评估（模型没有见过的数据）。

##Model Training

The current output label is categorical in nature, so it becomes a classification problem as discussed at the start.

>##模型训练

>当前的输出标签具备类别性，所以就成了开头讨论的分类问题。

Majority of the classification models are present in the sklearn library.

All you need to do is importing the model, instantiating the class with the required set of parameters.

After this step, you simply need to fit the model on the training set.

>大多数分类模型都存在于sklearn库中。

>你需要做的就是导入模型，用所需的参数集实例化类。

>在这一步之后，你只需要在训练集上拟合模型。

##Making predictions

>##做出预测

```
from sklearn.metrics import accuracy_score    
lr_pred_train = lr.predict(X_train) # prediction on the training set    
lr_pred_test = lr.predict(X_test) # prediction on the testing set    
dt_pred_train = dt.predict(X_train)    
dt_pred_test = dt.predict(X_test)    
print("----------LOGISTIC REGRESSION----------")    
print("Accuracy of Training Split :",accuracy_score(y_train, lr_pred_train))    
print("Accuracy of Test Split :",accuracy_score(y_test, lr_pred_test))    
print()    
print("----------DECISION TREE----------")    
print("Accuracy of Training Split :",accuracy_score(y_train, dt_pred_train))    
print("Accuracy of Test Split :",accuracy_score(y_test, dt_pred_test))
```

Once the model is trained, making predictions is very easy, use model.predict() function and in the parameter, provide the input features. Based on the model weights, the model will predict the corresponding output labels.

There are various metrics to evaluate the models, for this article we would be using accuracy_score.

If you want to know more refer to this [link](https://scikit-learn.org/stable/modules/model_evaluation.html#:~:text=3.3.%20Metrics%20and%20scoring%3A%20quantifying%20the%20...%201,comparing%20one%E2%80%99s%20estimator%20against%20simple%20rules%20of%20thumb. "link").

![](https://static.leiphone.com/uploads/new/sns/article/202011/1605507266748027.png)

As we could see above, even though both models perform very well on the training set, they fail to show the same promise for the testing result.

>从上面我们可以看出，即使这两个模型在训练集上的表现都非常好，但在测试结果上却没有表现出同样优秀的性能。

##Overfitting vs Underfitting

>##过拟合和欠拟合

A short story: Let’s say you gave 100 MCQ questions as the training set for the model, on which the model is trained. To test whether your model is working fine, you asked the same questions on which the model was trained.

Now, you might feel the model is doing great. How could you be assured that the model is answering based on knowledge or simple memorization of all answers. To tackle this, we use the testing set, something the model has not seen till now. The outcome of this evaluation would surely confirm whether the model is able to generalize well or has it purely memorized the 100 questions you gave to it.

In technical terms we call this as overfitting vs underfitting. A good model lies in between these 2 extreme conditions.

“A good Machine Learning Algorithm is the one which is able to generalize its observations and hence delivers excellent performance on new unseen data.”

To put it up practically,

>有一个小故事：假设你给了100个MCQ问题作为模型的训练集，模型就是在这些问题上训练出来的。为了测试你的模型是否运行良好，你又问了同样的问题，模型就是在这些问题上训练出来的。
直到现在，你可能也觉得模型做得很好。但是你怎么能保证模型是基于知识或简单记忆所有答案来回答的。为了解决这个问题，我们使用了测试集，这是模型到现在为止都没有见过的东西。这个评估的结果一定会证实模型是否能够很好的归纳，还是纯粹的记住了你给它的100道题。
在技术上我们称之为过拟合与欠拟合。一个好的模型介于这2个极端条件之间。

>"一个好的机器学习算法是能够将其观测结果泛化，从而在新的未见数据上提供优异的性能。"

>实际上:

In technical terms we call this as overfitting vs underfitting. A good model lies in between these 2 extreme conditions.

“A good Machine Learning Algorithm is the one which is able to generalize its observations and hence delivers excellent performance on new unseen data.”

To put it up practically,

Underfitting occurs when the accuracy on the training set itself is low.

Overfitting occurs when the accuracy on the training set is high but low on the testing set.

There are various ways to tackle these issues.

To tackle underfitting, try to increase the number of iterations of model training, increase the depth of tree based models.

If the above 2 methods don’t fetch satisfactory results, go for advanced feature extraction and feature engineering techniques.

PS: You could use KBest features, RFE(Recursive Feature Elimination etc)

For tackling overfitting, try to reduce the depth of tree based models, use regularization.

Mostly regularization would solve the issues else go for the advanced methods.

>当训练集本身的精度较低时，就会出现欠拟合。

>过拟合发生在训练集的准确率很高，但测试集的准确率很低的时候。

>解决这些问题的方法有很多种。

>要解决欠拟合，可以尝试增加模型训练的迭代次数，增加基于树的模型的深度。

>如果以上2种方法没有得到满意的结果，可以去使用更为先进的特征提取和特征工程技术。

>PS：你可以使用KBest特征、RFE(递归式特征消除等)

>对于解决过拟合的问题，尽量减少基于树的模型的深度，使用正则化。

>大部分正则化就能解决问题，否则就去找更先进的方法。

##Hyperparameter Tuning

>##调整超参

“No matter how good the performance is, there is always a scope of improvement”.

All the models have various hyperparameters that can be tweaked to optimize the performance. Hyperparameters are the parameters whose values are fixed manually before model training.

Examples include : “number of estimators”, “learning rate”, “no of splits” etc.

This could be a very time consuming step, as the model training takes place several times.

>"无论性能有多好，都有改进的空间"。

>所有模型都有各种超参数，可以通过调整其来优化性能。超参数是指在模型训练前手动固定其值的参数。

>例如：: "估算器的数量"、"学习率"、"不拆分 "等。

>这可能是一个非常耗时的步骤，因为模型训练需要多次进行。

```
# Import necessary modules    
from sklearn.linear_model import LogisticRegression    
from sklearn.model_selection import GridSearchCV    
# Setup the hyperparameter grid    
c_space = np.logspace(-5, 8, 15)    
param_grid = {'C': c_space}    
# Instantiate a logistic regression classifier: logreg    
logreg = LogisticRegression()    
# Instantiate the GridSearchCV object: logreg_cv    
logreg_cv = GridSearchCV(logreg, param_grid = param_grid, cv = 5)    
# Fit it to the data    
logreg_cv.fit(X, y)    
# Print the tuned parameters and score    
print("Tuned Logistic Regression Parameters: {}".format(logreg_cv.best_params_))    
print("Best score is {}".format(logreg_cv.best_score_))
```
Grid Search Cross Validation is used to find the optimal set of hyperparameters. In the param_grid which is a python dictionary, you can specify the various values to be tried on.

The outcome of the above code would be a model with the best set of hyperparameters amongst the provided values.

However the drawback of using GridSearchCV is that this method will go through all the intermediate combinations of hyperparameters making it computationally very expensive.

>Grid Search Cross Validation用于寻找最优的超参数集。在 param_grid（一个 python 字典）中，您可以指定要尝试的各种值。

>上述代码的结果将是在所提供的值中得到一个具有最佳超参数集的模型。

>然而，使用 GridSearchCV 的缺点是，该方法将通过所有超参数的中间组合，使其计算成本非常高。

```
# Import necessary modules    
from scipy.stats import randint    
from sklearn.tree import DecisionTreeClassifier    
from sklearn.model_selection import RandomizedSearchCV    
# Setup the parameters and distributions to sample from: param_dist    
param_dist = {"max_depth": [3, None],    
"max_features": randint(1, 9),    
"min_samples_leaf": randint(1, 9),    
"criterion": ["gini", "entropy"]}    
# Instantiate a Decision Tree classifier: tree    
tree = DecisionTreeClassifier()    
# Instantiate the RandomizedSearchCV object: tree_cv    
tree_cv = RandomizedSearchCV(tree, param_distributions = param_dist, cv=5, n_iter = 5)    
# Fit it to the data    
tree_cv.fit(X, y)    
# Print the tuned parameters and score    
print("Tuned Decision Tree Parameters: {}".format(tree_cv.best_params_))    
print("Best score is {}".format(tree_cv.best_score_))
```

Using Randomized Search CV solves the above issues as you can specify the value of “n_iter”. Since the value is currently 5, this means only 5 sets of randomly selected values will be tested.

This increases the chances to obtain the optimal model at the same time its computational complexity is based on the value of “n_iter”.

If you are running short of time, use lower value of “n_iter” through which the results would be obtained quickly, but the chances of getting the optimal set of hyperparameters would be less.

>使用随机搜索CV可以解决上述问题，因为你可以指定 "n_iter "的值。由于该值目前为5，这意味着只有5组随机选择的值将被测试。

>这增加了获得最佳模型的机会，同时其计算复杂度是基于 "n_iter "的值。

>如果你的时间有限，使用较低的 "n_iter "值，这样可以快速得到结果，但得到最优超参数集的几率会更小。

##Bonus Part-1 : Creating a pipeline

You might be wondering that since the steps are pretty much sequential, scaling the data and then fitting the model and hyperparameter tuning, why not club them together.

>##奖励部分-1：创建一个管道

>你可能会觉得奇怪，既然步骤很有顺序，缩放数据，然后拟合模型和超参数调优，为什么不把它们连在一起。

```
# Setup the pipeline    
steps = [('scaler', StandardScaler()),    
('SVM', SVC())]    
pipeline = Pipeline(steps)    
# Specify the hyperparameter space    
parameters = {'SVM__C':[1, 10, 100],    
'SVM__gamma':[0.1, 0.01]}    
# Create train and test sets    
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 21)    
# Instantiate the GridSearchCV object: cv    
cv = GridSearchCV(pipeline, param_grid = parameters, cv = 3)    
# Fit to the training set    
cv.fit(X_train, y_train)    
# Predict the labels of the test set: y_pred    
y_pred = cv.predict(X_test)    
# Compute and print metrics    
print("Accuracy: {}".format(cv.score(X_test, y_test)))    
print(classification_report(y_test, y_pred))    
print("Tuned Model Parameters: {}".format(cv.best_params_))
```
With the help of sklearn pipeline, all of the steps could be combined into a single step.

I already told you, sklearn is an awesome library with lots of awesome functionalities.

>在sklearn pipeline的帮助下，所有的步骤都可以合并成一个步骤。

>我已经告诉过你了，sklearn是一个有着很多很棒功能的库。

##Bonus Part-2: Ensemble Learning

In almost every competition, ensemble techniques fetch the best results.

Ensemble learning in simplistic terms means “together”. There’s a common saying, “a crowd is clever together than a clever individual”.

The main causes of error in learning models are due to noise, bias and variance. Ensemble methods help to minimize these factors. Ensemble methods are designed to improve the stability and the accuracy of Machine Learning algorithms.

So here we basically train lots of weak learners and the final result/prediction is taken up by aggregation of the predictions by individual weak learning or by the process of voting akin to real life elections.

>##奖励部分-2：集成学习

>在几乎所有的比赛中，集合技术都能取得最好的成绩。

>集成学习简单来说就是 "一起"。有句俗话说："一群人在一起比一个人聪明"。

>在学习模型中，产生误差的主要原因是由于噪声、偏差和方差。集成方法有助于最小化这些因素。集成方法是为了提高机器学习算法的稳定性和准确性。

>所以在这里，我们基本上是训练了很多weak learners，最后的结果/预测是由各个learners的预测汇总或者类似于现实生活中选举的投票过程。

##Voting    (投票)

```
from sklearn.linear_model import LogisticRegression    
from sklearn.neighbors import KNeighborsClassifier    
from sklearn.tree import DecisionTreeClassifier    
from sklearn.ensemble import VotingClassifier    
from sklearn.metrics import f1_score, classification_report    
# Instantiate the individual models    
clf_knn = KNeighborsClassifier(n_neighbors = 5)    
clf_lr = LogisticRegression(class_weight = 'balanced')    
clf_dt = DecisionTreeClassifier(min_samples_leaf = 3, min_samples_split = 9, random_state=500)    
# Create and fit the voting classifier    
clf_vote = VotingClassifier(    
estimators=[('knn', clf_knn), ('lr', clf_lr), ('dt', clf_dt)]    
)    
clf_vote.fit(X_train, y_train)    
# Calculate the predictions using the voting classifier    
pred_vote = clf_vote.predict(X_test)    
# Calculate the F1-Score of the voting classifier    
score_vote = f1_score(y_test, pred_vote)    
print('F1-Score: {:.3f}'.format(score_vote))    
# Calculate the classification report    
report = classification_report(y_test, pred_vote)    
print(report)
```

Here, you can combine three models together to fetch the final prediction based on voting.

I find this method pretty useful as we usually train atleast 3 models to have a comparison based on performance. Combining the 3 or more best models together could result into better results.

The performance might not be better, but is at least as good as the best individual model — the decision tree.
This may be because the other two models had a performance around 10% lower than the decision tree.

Voting Classifier is used when the target variable is categorical.
By default voting is ‘hard’. Use the parameter voting = ‘soft’, if you want the average of the predicted probabilities

>在这里，你可以将三个模型组合在一起，根据投票情况来获取最终的预测。

>我觉得这个方法非常有用，因为我们通常会训练至少3个模型来进行性能的比较。将3个或更多最好的模型组合在一起可以得到更好的结果。

>性能可能不会更好，但至少和最好的单个模型—决策树一样好。

>这可能是因为其他两个模型的性能比决策树低10%左右。

>当目标变量是分类类别时，会使用投票分类器。
默认情况下，投票是'hard'。如果你想要预测概率的平均值，请使用参数 voting = 'soft'。

**A few other models to use**

Random forest model could be used which is an ensemble of decision trees. It is the most commonly used algorithm.

Alternatively, you could make use of :

Gradient Boosting classifier

>**其他一些可用模型**

>可以使用随机森林模型，这是一个决策树的集合。这是最常用的算法。或者可以使用：

>梯度提升分类器

```
from sklearn.ensemble import GradientBoostingClassifier    
# Build and fit a Gradient Boosting classifier    
clf_gbm = GradientBoostingClassifier(n_estimators = 100, learning_rate = 0.1, random_state=500)    
clf_gbm.fit(X_train, y_train)    
# Calculate the predictions on the test set    
pred = clf_gbm.predict(X_test)    
# Evaluate the performance based on the accuracy    
acc = accuracy_score(y_test, pred)    
print('Accuracy: {:.3f}'.format(acc))    
# Get and show the Confusion Matrix    
cm = confusion_matrix(y_test, pred)    
print(cm)
```
Ada Boost Classifier

>AdaBoost分类器

```
from sklearn.ensemble import AdaBoostClassifier    
from sklearn.tree import DecisionTreeClassifier    
dt = DecisionTreeClassifier()    
# Build and fit an AdaBoost regressor    
ada = AdaBoostClassifier(base_estimator = dt, n_estimators = 12, random_state=500)    
ada.fit(X_train, y_train)    
# Calculate the predictions on the test set    
pred = ada.predict(X_test)
```
Categorical Boosting

```
import catboost as cb    
# Build and fit a CatBoost Classifier    
cat = cb.CatBoostClassifier(n_estimators = 100, learning_rate = 0.1, max_depth = 3, random_state=500)    
cat.fit(X_train, y_train)    
# Calculate the predictions on the testing set    
pred = reg_cat.predict(X_test)
```

Here, the default base estimator is decision tree. There is no need to encode the categorical features. The model could be fit directly.

>这里，默认的基础估算器是决策树。不需要对分类特征进行编码。模型可以直接拟合。

本作品采用知识共享署名-非商业性使用-相同方式共享 4.0 国际许可协议进行许可，转载请附上原文出处链接和本声明。
本文链接地址：https://flyai.com/article/771

讨论

500字

表情

发送

删除确认

是否删除该条评论？

取消删除