In the classification of cancer data sets, we note that they contain a number of additional features that influence the classification accuracy. There are many evolutionary algorithms that are used to define the feature and reduce dimensional patterns such as the gray wolf algorithm (GWO) after converting it from a continuous space to a discrete space. In this paper, a method of feature selection was proposed through two consecutive stages in the first stage, the fuzzy mutual information (FMI) technique is used to determine the most important feature selection of diseases dataset through a fuzzy model that was built based on the data size. In the second stage, the binary gray wolf optimization (BGWO) algorithm is used to determine a specific number of features affecting the process of classification, which came from the first stage. The proposed algorithm, FMI_BGWO, describes efficiency and effectiveness by obtaining a higher classification accuracy and a small number of selected genes compared to other competitor algorithms.

Mutual information (MI) is one of the filtering methods, which is calculated between two random variables using entropy. Entropy measures an average random variable the amount of information required to describe the random variable [1].

Gray wolf optimization (GWO) is a swarm intelligent technique developed by Mirjalili, et al. [2]. The hunting techniques and the social hierarchy of wolves are mathematically modeled in order to develop GWO and perform optimization. The GWO algorithm is tested with the standard test functions that indicate that it has superior exploration and exploitation characteristics than other swarm intelligence techniques [3]. Scientists have developed the basic algorithm GWO by converting the search algorithm from a continuous search space to a discrete search space. This modified algorithm is called a binary gray wolf optimization(BGWO) algorithm, which works in binary search areas and uses binary values ​​equal to 0 or 1 [3,4].

Feature subset selection method is a procedure to reduce the number of unnecessary feature from the original feature set [5,6]. This method is used when we have a large number of features in a dataset, where the method of determining the feature to take a number of features necessary. The feature selection can be applied in several areas and is intended to improve the accuracy of the classification. Two basic methods are used in selecting the feature: filtering approaches, wrapper approaches.

In this study, a new FMI_BGWO algorithm is suggested to determine the best-feature selection. Our proposed algorithm can efficiently achievement the powerful points of both FMI and BGWO algorithms in finding the most important feature. The experimental results show the excellent performance of the proposed algorithm when the number of feature is high, and the sample size is low.

The remains of this paper are organized as follows. The GWO is presented in Section 2. In Sections 3 and 4, a brief description of feature selection and MI respectively. In Section 5, the proposed algorithm is explained. Section 6 covers the obtained results and their discussion. Finally, the most important general conclusions are mentioned, in section 7.

Grey Wolf Optimization (GWO)

The gray wolf (Canis lupus) is part of the Canidae family. Gray wolves are types with a very rigorous social dominant hierarchy of leadership. Males and females in the pack are leaders, and they are called alpha [2]. Alpha wolf is also known as the dominant wolf because all its orders must be followed by all wolves in the pack. Only alpha wolves are allowed to mate in the pack. This means that the order of the pack and its regularity is more important than its strength. Beta is the second level in the hierarchy of gray wolves.

Beta are wolves that help alpha wolves make decisions or other things about the pack. The wolf beta is the best candidate to be alpha in the case if one of the wolves of alpha becomes too big or in the case of the death of someone in command.

Omega is less gray wolves. The omega wolf always offers all the other dominant wolves on it. Omega acts as a scapegoat. The last wolves allowed to eat are omega. We may apparently notice that omega is not an important member of the group, but in fact, if omega is lost, the entire pack will face problems and internal fighting.

Delta is the fourth class in the pack. Delta wolves control omega, but they must undergo alpha and beta orders. Delta shall be responsible for guards, scouts, hunters, elders, and patients and each has its own specific responsibility [7].

Mathematical modelling

The first best solution in GWO is alpha (α), the second-best solution is beta (β), and the third-best solution is Delta (δ). While the rest of the solutions are Omega (ω), which follows the rest of the first three solutions [8].

Encircling prey: Gray wolves surrounded prey while hunting. The following equations will represent a mathematical model about the surrounding behavior:

$\overrightarrow{X}\left(\text{t}+1\right)=\overrightarrow{X_p}\left(t\right)- \overrightarrow{A}\cdot \overrightarrow{D} (1)$

$\overrightarrow{D}=\left|\overrightarrow{C}\cdot \overrightarrow{X_p}\left(t\right)-\overrightarrow{X}\left(t\right)\right| \left(2\right)$

where t is the iteration, $\overrightarrow{X_p}$ is the prey position, $\overrightarrow{X}$ is the gray wolf position,

the $\cdot$ operator indicates vector entry wise multiplication.

Where $\overrightarrow{A}$ and $\overrightarrow{C}$ are coefficient vectors calculated as follows:

$\overrightarrow{A}=2\overrightarrow{a}\cdot \overrightarrow{r_1}-a (3)$

$\overrightarrow{C}=2 \overrightarrow{r_2}(4)$

where components of $\overrightarrow{a}$ are linearly decreased from 2 to 0 over the course of iterations and r1, r2, are random vectors in [0,1].

Hunting: Gray wolves have the ability to encircle prey and locate them. Hunting is performed by a complete pack based on, Information from the alpha, beta and delta wolves, so the updating for the wolves positions is as in the following equations:

$\overrightarrow{X}\left(t+1\right)=\frac{\overrightarrow{X_1}+\overrightarrow{X_2}+\overrightarrow{X_3}}{3} (5)$

$\overrightarrow{X_1}=\left|\overrightarrow{X_{\alpha }}-\overrightarrow{A_1}·\overrightarrow{D_{\alpha }}\right| , \overrightarrow{X_2}=\left|\overrightarrow{X_{\beta }}-\overrightarrow{A_2}·\overrightarrow{D_{\beta }}\right| , \overrightarrow{X_3}=\left|\overrightarrow{X_{\delta }}-\overrightarrow{A_3}·\overrightarrow{D_{\delta }}\right|(6)$

$\overrightarrow{D_{\alpha }}=\left|\overrightarrow{C_1}·\overrightarrow{X_{\alpha }}-\overrightarrow{X}\right|,\overrightarrow{D_{\beta }}=\left|\overrightarrow{C_2}·\overrightarrow{X_{\beta }}-\overrightarrow{X}\right|,\overrightarrow{D_{\delta }}=\left|\overrightarrow{C_3}·\overrightarrow{X_{\delta }}-\overrightarrow{X}\right|(7)$

The updating of the parameter $\overrightarrow{a}$ through the following equation:

$\overrightarrow{a}=2-t\cdot \frac{2}{Ma{x}_{iter}}(8)$

Where t is the iteration number and Max_iter is the total number of iteration allowed for the optimization [9]. The pseudo code of the GWO algorithm is displayed in figure 1.

Binary Gray Wolf Optimization (BGWO)

The positions of gray wolves constantly change in space to any point. Solutions are limited to binary values such as 0 or 1 in some special problems such as feature selection. In our research, we suggested feature selection through a binary GWO algorithm. Updating the equations of wolves are a function of the positions of three vectors is, X_a, X_β and X_ε which represents the best three solutions and which attracts each wolf towards it. At any given time, all solutions are at a corner of a hypercube and the solutions are grouped in binary form. Based on the basic GWO algorithm, the given wolf positions are updated, a binary restriction must be maintained according to equation (9).

The main update equation in the GWO algorithm is written as [3]:

$X_i^{t+1}=Crossover\left(X_1,X_2,X_3\right)(9)$

Where (X₁, X₂, X₃) they are binary vectors representing wolves in bGWO and crossover (a, b, c) are an appropriate intersection between solutions (a, b, c) and (X₁, X₂, X₃), and the wolves calculate alpha, beta and delta in order using equations (10), (13) and (16).

$X_1^d=\{\begin{array}{c}1 if (X_{\alpha }^d+bste{p}_{\alpha }^d)\ge 1\\ 0 other wise \end{array}(10)$

Where $bste{p}_{\alpha }^d$ is a binary step in dimension d, $X_{\alpha }^d$ is a vector representing the position of the alpha wolf in dimension d. $bste{p}_{\alpha }^d$ is calculated by the following equation:

$bste{p}_{\alpha }^d=\{\begin{array}{c}1 if cste{p}_{\alpha }^d\ge rand\\ 0 other wise\end{array} (11)$

where $bste{p}_{\alpha }^d$ is the continuous-valued step size for dimension d, rand is a random number in the closed period [0,1] derived from the uniform distribution. $bste{p}_{\alpha }^d$ can be calculated using the sigmoidal function by the following equation:

$cste{p}_{\alpha }^d=\frac{1}{1+e^{-10(A_1^d D_{\alpha }^d-0.5)}} (12)$

Where $A_1^d and D_{\alpha }^d$ are calculated using equations (3), and (5) in the dimension d.

$X_2^d=\{\begin{array}{c}1 if (X_{\beta }^d+bste{p}_{\beta }^d)\ge 1\\ 0 other wise\end{array} (13)$

Where $bste{p}_{\alpha }^d$ is a binary step in dimension d, $bste{p}_{\alpha }^d$ represents the wolf position beta vector in dimension d. $bste{p}_{\alpha }^d$ is calculated by the following equation:

$bste{p}_{\beta }^d=\{\begin{array}{c}1 if cste{p}_{\beta }^d\ge rand\\ 0 other wise\end{array} (14)$

Where $cste{p}_{\beta }^d$ is the step size with a continuous value of dimension d, and (rand) is a random number derived from the uniform distribution in the closed period [0,1].

$cste{p}_{\beta }^d$ is calculated using the sigmoidal function by equation:

$cste{p}_{\beta }^d=\frac{1}{1+e^{-10(A_1^d D_{\beta }^d-0.5)}} (15)$

Where $A_1^d and D_{\beta }^d$ is the step size with a continuous value of dimension d, and (rand) is a random number derived from the uniform distribution in the closed period [0,1].

$A_1^d and D_{\beta }^d$ is calculated using the sigmoidal function by equation:

Where $A_1^d and D_{\beta }^d$ are calculated using equations (3), and (5) in the dimension d.

$X_3^d=\{\begin{array}{c}1 if (X_{\delta }^d+bste{p}_{\delta }^d)\ge 1\\ 0 other wise \end{array} (16)$

Where $(X_{\delta }^d)$ represents the location of the wolf delta (δ) in dimension d, and $bste{p}_{\delta }^d$ is a binary step in dimension d calculated by:

$bste{p}_{\delta }^d=\{\begin{array}{c}1 if cste{p}_{\delta }^d\ge rand\\ 0 other wise \end{array}(17)$

Where rand is a random number derived from the uniform distribution in the closed period [0,1], and is the step size with a continuous value of dimension d and can be calculated using the sigmoidal function by Eq. (18).

$cste{p}_{\delta }^d=\frac{1}{1+e^{-10(A_1^d D_{\delta }^d-0.5)}}(18)$

Where $A_1^d and D_{\beta }^d$ are calculated using equations (3), and (5) in the dimension d.

In the following equations, we will apply the intersection to each of the solutions a, b, c:

$X_d=\{\begin{array}{c}{a}_{d } if rand<1/3\\ b_d if 1/3\le rand<2/3\\ c_{d } other wise \end{array} (19)$

where rand is a random number derived from the uniform distribution in the closed period [0,1], and a_d, b_d and c_d represents the binary values of the first, second and third parameters of the dimension d, X_d is the result of the exchange in dimension d.

BGWO algorithm is displayed in figure 2 [10]:

Naïve bayes classifier

Naïve Bayes is one of the great models used in the classifications, which calculates the probability that a particular feature belongs to a particular category it is assumed that the constituent features of the search space are restricted and dependent on certain conditions [10]. Naïve Bayes performs mostly well in terms of simplicity of construction and ease of implementation. From the example that describes the feature vector which is (X₁, x₂, …,x_n), we will look for the class G which increases the probability of:

$P \left(\frac{X}{G}\right)=P\text{(}{x}_1,x_2,\dots ,x_n\text{|}g) (20)$

The Naïve Bayes method will allow conditional independence between the features in the data by expressing this probability $P \left(\frac{X}{G}\right)$ as a result of those probabilities [11,12]:

$P\left(\frac{X}{G}\right)={\pi }_{i=1}^{n}P\left(\frac{X}{G}\right) (21)$

The feature selection method improves the performance of the algorithm by reducing the number of attributes used to describe a data set [6,13]. The purpose of features selecting in an algorithm is to reduce the number of genes used to improve classification and increase classification accuracy [14]. The feature identification algorithms consist of three parts:

1. Search algorithm: A subset of properties (features), which are part of the original features.

2. Fitness function: these input and digital assessment modes. The goal of the search algorithm is to draw attention to this function.

3. Classifier: It represents the required algorithm that uses the latest subset of genes (i.e., an algorithm that selects the most important features required) [15,16] (Figure 3).

One of the ways to select a feature on the predictive performance of a predefined algorithm is the wrapper method, which evaluates the quality and efficiency of certain features [17]. This is based on two-steps, based on a specific algorithm: (1) Searching for a subset of properties and features (2) Evaluate these specific features and attributes. In order to obtain learning performance or to reach some stop criteria we repeat (1) and (2) [12,18].

Mutual Information (MI)

Mutual information technology is widely used in feature selection and also in Sequence alignments. Mutual information technology arranges features according to their importance to data [12]. Mutual information technology is used to find out the relationship between some features and category classifications through the following relationship:

$MutualInfo\left(X_i,C\right)={\displaystyle \sum }_{c\in C}{\displaystyle \sum }_{x_i\in X_i}P(X_i,y)\log \frac{P(X_i,y)}{P(X_i)P(y)} (22)$

Where C is the set of categories, and X_i is the set of features in position i.

In mutual information technology, if there is a close correlation between the category and the feature, then the values of the mutual information will be large, but in the absence of a correlation between the feature and the category, the mutual information = 0. The mutual information can also be described as the degree of uncertainty in a feature that has been removed by knowing its category:

$\text{MutualInfo}\left({\text{X}}_{\text{i}},\text{C}\right)=\text{H}\left({\text{X}}_{\text{i}}\right)-\text{H}\left({\text{X}}_{\text{i}}\backslash \text{C}\right) (23)$

Where H(Xi) is an entropy in X_i and H(X_i\C) is an X_i entropy after knowing C so if knowledge of C provides a lot of information then H(X_i\C) will be low and therefore MutualInfo will be high. One disadvantage of the information exchanged favors rare features if the conditional probabilities are equal to P(X_i,y) and this may not always be the desired result [19].

The proposed algorithm

The proposed method for FMI _BGWO is two basic stages. In the first stage, the fuzzy mutual information (FMI) method is used to determine the most important feature based on a fuzzy Mam dani model created by relying on two input variables that represent the size of the sample data and the number of genes per sample. Three linguistic variables (high, mean and low) were selected for both input and output. The output range was also fixed between (10-50) to select the number of genes arranged by the MI method and which will be submitted to the second phase. In the second stage, the BGWO algorithm is used to reduce and define a certain number of features, which came from the first stage. In the BGWO method, the fitness function (FF) is used to evaluate wolves’ positions as follows:

$FF=0.1*C+0.9*\left(\frac{q}{p-q}\right) (24)$

Where (q) is the length of the selected gene subset, (p) is the total number of features, (c) is the accuracy of the classification model. The vector in figure 2 consists of a series of binary values 0 and 1 that represents a subset of features, in this vector, all features can be selected.

The second stage of the proposed algorithm FMI_BGWO concentrates on the BGWO, specifically the wrapper in feature selection (Figure 4).

Through the method (FMI) obtained the most important features, these features will be entered in the algorithm (BGWO) to reduce them as a second stage. Using an algorithm (BGWO), a certain number of features are arranged and selected in the final classification process using the Naive Bayes classifier. Figure 5 presents an elaborate flowchart of the proposed FMI_BGWO structure.