Sequential drug decision problems in long-term medical conditions: a case Study of Primary Hypertension Eunju Kim ba, ma, msc

WHILE The stopping rule is satisfied.

Pick a new solution π’ randomly in the neighbourhood of the current solution (N(π)).

IF f(π’) > f(π) for a maximisation problem (f(π’) < f(π) for a minimisation problem).

THEN Replace the current solution π with the new solution π’.

ELSE Accept the new solution π’ with (Equation 3.3).

END

Update the temperature T.

LOOP

Figure ‎3.. Pseudo-code of SA^[208]

3.4Genetic algorithm

GA is a computational model of evolutionary processes, which mimics natural biological evolution. GA applies the principle of survival of the fittest on a population of potential solutions encoded as a form of chromosomes to produce a solution^[179]. The first step in the GA is to generate an initial population including n individuals (i.e., n sequential treatment policies in SDDPs) (see Figure ‎3.). Small populations may run the risk of seriously under-covering the solution space, while large populations incur severe computational cost. The ideal size of population is problem-specific. It has been varied from 8^[211] to 3000^[100]. Reeves said that population sizes as small as 30 are adequate in many cases^[53].

Individuals are usually encoded in a string of standard binary (0,1) digits^[211-213], real numbers^{[85, 214]} or multiple characters^[215]. The assumption is that the components of individuals (which are called chromosomes) represent the genetic structure of the parent, and the superior elements are encouraged to find their way into the chromosomes of the offspring. At each generation, a new set of chromosomes is created by selecting individuals according to their level of fitness in the problem domain (i.e., the value of an objective function or the result of a simulation experiment) and then breeding them together using a number of genetic operators.

The selection operator selects certain number of chromosomes for the succeeding generation from the current population. A simple approach, which is called the elitist method, uses the value of the objective function associated with each chromosome^{[211, 216]}. The fittest individuals are deterministically allowed to propagate through successive generations. Although the elitist method guarantees the preservation of the current best chromosomes, it is not usually recommended because of the risk of premature convergence to a poor local optimum^[53]. As an alternative, the tournament selection method compares two individuals selected at random and then selects the best as a parent until a certain number of parents are selected in this way^{[85, 214]}. The roulette wheel selection method selects the individuals according to their probabilities of selection based on their fitness^{[100, 217, 218]}. That is, individuals with a higher fitness have a higher probability to be chosen as members of the population of the next iteration.

After the selection step, the selected population are paired into parents. A crossover operator splits the parent chromosomes at a chosen splice point and then swaps all data beyond that point with the other chromosomes to create offspring. The split point can be one, which is called single-point crossover, or multiple, which is called multi-point crossover. The crossover point is often selected by sampling from a uniform probability distribution^{[211, 213, 217]}. Crossover rate, which is the probability of crossover occurring between pairs of individuals, ranged from 0.4%^[211] to 1.0%^[213] in the studies included in the systematic review^{[211, 213, 215]}.

Mutation causes a random change of one or more bits on the offspring’s chromosomes with low probability in order to prevent the search from diversifying too rapidly, typically in the range 0.1% and 0.5%^{[213, 214]}. In this way GA reduces the dependence of the solution on its starting chromosomes, but preserves the good chromosomes created through crossover. When the offspring is infeasible, the infeasible offspring is handled by rejecting or repairing the infeasible solutions or applying a penalty function. GA generally terminates after a pre-specified number of generations. If no acceptable solutions are found, the GA may be restarted or a fresh search initiated.

Generate the initial population P. Evaluate the individuals in the initial population. WHILE The stopping rule is satisfied. Select the superior individuals from the current population. Recombine the elements of the parents’ chromosomes to produce new individuals. Replace each element of the new individuals with a low probability. Evaluate the individuals in the current population. LOOP

Figure ‎3.. Pseudo-code of GA^[208]