40 Questions to ask a Data Scientist on Ensemble Modeling Techniques (Skilltest Solution)

1. Different learners can come from same algorithm with different hyper parameters
2. Different learners can come from different algorithms
3. Different learners can come from different training spaces

A. 1
B. 2
C. 1 and 3
D. 1, 2 and 3

A. True
B. False

Q

Note: All individual models have meaningful and good predictions.

A. True
B. False

Solution: (B)

An ensemble is an art of combining a diverse set of learners (individual models) together to improvise on the stability and predictive power of the model. So, creating an ensemble of diverse models is a very important factor to achieve better results.

1. They have low variance and they don’t usually overfit
2. They have high bias, so they can not solve hard learning problems
3. They have high variance and they don’t usually overfit

A. 1 and 2
B. 1 and 3
C. 2 and 3
D. None of these

Solution: (A)
Weak learners are sure about particular part of a problem. So they usually don’t overfit which means that weak learners have low variance and high bias.

A. True

B. False

Solution: (A)

Usually, ensemble would improve the model, but it is not necessary. Hence, option A is correct.

A. Yes
B. No
C. can’t say

Solution: (B)

It is not necessary. Ensemble of weak learners can also yield a good model.

A. Less correlation among predictions
B. High correlation among predictions
C. Correlation does not have any impact on ensemble output
D. None of the above

Solution: (A)

A lower correlation among ensemble model members will increase the error-correcting capability of the model. So it is preferred to use models with low correlations when creating ensembles.

A. Bagging
B. Boosting
C. A Or B
D. None of these

Solution: (A)

In bagged ensemble, the predictions of the individual models won’t depend on each other. So option A is correct.

1. Median
2. Product
3. Average
4. Weighted sum
5. Minimum and Maximum
6. Generalized mean rule

A. 1, 3 and 4
B. 1,3 and 6
C. 1,3, 4 and 6
D. All of above

Solution: (D)

All of the above options are valid methods for aggregating results of different models (in case of a regression model).

A. 100%
B. 78.38 %
C. 44%
D. 70

Solution: (A)

Refer below table for models M1, M2 and M3.

Solution: (C)

Refer below table for models M1, M2 and M3.

1.  Use an algorithm to return the optimal weights
2. Choose the weights using cross validation
3. Give high weights to more accurate models

A. 1 and 2
B. 1 and 3
C. 2 and 3
D. All of above

Solution: (D)

All of the options are correct to decide weights of individual models in an ensemble.

A. It can only be used in classification problem
B. It can only be used in regression problem
C. It can be used in both classification as well as regression
D. None of these

Solution: (C)

You can use average ensemble on classification as well as regression. In classification, you can apply averaging on prediction probabilities whereas in regression you can directly average the prediction of different models.

A. [ 0., 0.66666667, 0.33333333, 1. ]

B. [ 0.1210, 0.66666667, 0.95,0.33333333 ]

C. [ 0.1210, 0.66666667, 0.33333333, 0.95 ]

D. None of above

Solution: (A)

The following steps can be applied to get the output in options A

1. Given ranking to the predictions
2. Scale these ranking using min max scaling

You can follow this code in python to get the desired result.

from sklearn.preprocessing import MinMaxScaler
x = np.array([0.2,0.5,0.33, 0.8])
sc = MinMaxScaler()
sc.fit_transform(np.argsort(x))

A) A
B) B
C) C
D) D
E) E

Solution: (C)

A. 1 and 3
B. 2 and 3
C. 1 and 2
D. 1, 2 and 3
E. None of above

Solution: (D)

All of the statements are true.

Note: You are applying the averaging method to ensemble given predictions by three models.

A.

B.

C.

D. None of these

Solution: (B)

Take the average of predictions of each models for each observation then apply threshold 0.5 you will get B answer.

For example, in first observation of the models (M1, M2 and M3) the outputs are 0.70,0.80,0.75 if you take the average of these three you will get 0.75 which is above 0.5 that means this observation will belong to class 1.

B.

C.

D. None of these

Solution: (B)

Take the weighted average of the predictions of each model for each observation then apply threshold 0.5 you will get B answer.

For example, in first observation of models (M1,M2 and M3) the outputs are 0.70,0.80,0.75 if you take the weighted average of these three predictions, you will get 0.745 output (0.70 * 0.4 + 0.80 *0.3 + 0.75* 0.3) which is above 0.5 that means this observation will belong to class 1.

Context: Question 22-23

Suppose in binary classification problem, you have given the following predictions of three models (M1, M2, M3) for five observations of test data set.

B.

C.

D. None of these

Solution: (B)

Take the majority voting for the predictions of each model for each observation.

For example for a first observation of models(M1,M2 and M3) the outputs are  1,1,0 if you take the majority voting of these three models predictions you will get 2 votes for class 1 that means this observation will belong to class 1.

Hint: Count the vote of M1,  M2, and M3 as 2.5 times, 6.5 times and 3.5 times respectively.

A.

B.

C.

D. None of these

Solution: (C)

Follow the steps in question number 20,21 and 22.

1. A machine learning model is trained on predictions of multiple machine learning models
2. A Logistic regression will definitely work better in the second stage as compared to other classification methods
3. First stage models are trained on full / partial feature space of training data

A.1 and 2

B. 2 and 3

C. 1 and 3

D. All of above

Solution: (C)

1. In stacking, you train a machine learning model on predictions of multiple base models.
2. It is not necessary – we can use different algorithms for aggregating the results.
3. First stage models are trained on all the original features.

• More robust model
• better prediction
• Lower time of execution

A. 1 and 2

B. 2 and 3

C. 1 and 3

D. All of the above

Solution: (A)

Option 1 and 2 are advantages of stacking whereas option 3 is not correct as staking takes higher time.

B.

C.  None of these

Solution: (A)

A is correct because it is aggregating the results of base models by applying a function f (you can say a model) on the outputs of d1, d2 and dL.

1. Divide the training data into k folds
2. Train k models on each k-1 folds and get the out of fold predictions for remaining one fold
3. Divide the test data set in “k” folds and get individual fold predictions by different algorithms

A. 1 and 2
B. 2 and 3
C. 1 and 3
D. All of above

Solution: (A)

The third option is not correct because we don’t create folds for test data in stacking.

A. Stacking has less stable CV compared to Blending
B. In Blending, you create out of fold prediction
C. Stacking is simpler than Blending
D. None of these

Solution: (D)

Only option D is correct.

• Here, we are working on binary classification problem
• All base models are trained on all features
• You are using k folds for base models

A. You will have only k features after the first stage
B. You will have only m features after the first stage
C. You will have k+m features after the first stage
D. You will have k*n features after the first stage
E. None of the above

Solution: (B)

If you have m base models in stacking. That will generate m features for second stage models.

1. Bagging can be parallel
2. The aim of bagging is to reduce bias not variance
3. Bagging helps in reducing overfitting

A. 1 and 2
B. 2 and 3
C. 1 and 3
D. All of these

Solution: (C)

1. In bagging individual learners are not dependent on each other so they can be parallel
2-3 The bagging is suitable for high variance low bias models or you can say for complex models.

A. True
B. False

Solution: (B)

In boosting you always try to add new models that correct previous models weaknesses. So it is sequential.

Q32. Below are the  two ensemble models:
1. E1(M1, M2, M3) and
2. E2(M4, M5, M6)

Above, Mx is the individual base models.

Which of the following are more likely to choose if following conditions for E1 and E2 are given?

E1: Individual Models accuracies are high but models are of the same type or in another term less diverse
E2: Individual Models accuracies are high but they are of different types in another term high diverse in nature

A. E1
B. E2
C. Any of E1 and E2
D. None of these

Solution: (B)

We have to select E2  because it contains diverse models. So option B is a correct option.

Solution: (C)

You can apply both the algorithms. In step wise forward selection, you will start with empty predictions and will add the predictions of models one at a time if they improve the accuracy of an ensemble.  In Step wise backward elimination, you will start with full set of features and remove model predictions one by one if after removing the predictions of model give an improvement in accuracy.

1. Add the models predictions (or in another term take the average) one by one in the ensemble which improves the metrics in the validation set.
2. Start with empty ensemble
3. Return the ensemble from the nested set of ensembles that has maximum performance on the validation set

A. 1-2-3
B. 1-3-4
C. 2-1-3
D. None of above

Solution: (C)

Option C is correct.

Solution: (B)

Because in dropout, weights are shared and the ensemble of subnetworks are trained together.

B. 9

C. 12

D. 16

E. None of the above

Solution: (B)

There are 16 possible combinations, of which only 9 are viable. Non-viable are (6, 7, 12, 13, 14, 15, 16).

B. Model capacity decreases in increase in dropout rate

C. Model capacity is not affected on increase in dropout rate

D. None of these

Solution: (B)

The subnetworks have more number of neurons to work with when dropout rate is low. So They are more complex resulting in an increase in overall model complexity. Refer chap 11 of DL book.

1. Max number of samples
2. Max features
3. Bootstrapping of samples
4. Bootstrapping of features

A. 1 and 3
B. 2 and 4
C. 1,2 and 3
D. 1,3 and 4
E. All of above

Solution: (E)

All of the techniques given in the options can be applied to get the good ensemble.

A. True
B. False

Solution: (A)

Refer the introduction part of this paper.

A. 0.05
B. 0.06
C. 0.07
D. 0.09

Ans: B

Solution:

Refer this link