Sequential drug decision problems in long-term medical conditions: a case Study of Primary Hypertension Eunju Kim ba, ma, msc

6.5Genetic algorithm

In the context of the hypertension SDDP, a chromosome, mChrom, represents a sequential treatment policy. For convenience, all chromosomes (i.e., sequential treatment policy numbers from 1 to 4,128) were stored as 2^13 binary data, which uses a string of 0s and 1s. For example, the policy number 1, which is the combination of Ds, BBs, CCBs and ACEIs/ARBs, was converted to ‘0-0-0-0-0-0-0-0-0-0-0-0-1’; and the policy number 4,128, which is the combination of ACEIs/ARBs, CCBs+ACEIs/ARBs, BBs+CCBs+ACEIs/ARBs and Ds+CCBs+ACEIs/ARBs, was represented as ‘1-0-0-0-0-0-0-1-0-0-0-0-0’. To evaluate the objective function, the individual strings were decoded by phenotypes (i.e., mPhen), which were the policy numbers included in the population.

Whereas SA moves from one solution to another, GA works by maintaining a population of solutions whose fitness values (i.e., the value of the objective function) are better. Small populations have the risk of seriously under-covering the solution space, whereas large populations require increased computational time. In the hypertension SDDP model, various combinations of the number of generations (nGeneration = 25, 50 and 100) and population size (nInitPop = 10, 30, 50 and 100) were tested to determine the best set of the number of generations and the population size for the SDDP in primary hypertension. The size of the population was assumed to be fixed, whereas some studies assumed a variable population size^{[365, 366]}.

The hypertension SDDP model selected the initial population using a random number generator that uniformly distributes numbers between 1 and 4,128. During every iteration, the individuals (i.e., sequential treatment policies) included in the population were evaluated in terms of the fitness function, ObjVal, which is the mean total net benefit based on 100 replications from the underlying evaluation model.

The applied GA used the roulette wheel selection method, which selects individuals probabilistically based on their fitness (see Figure ‎6.). An individual’s slice of a Monte Carlo-based roulette wheel is an area proportional to its fitness^{[217, 218]}. The fittest individual has the largest share of the roulette wheel, whereas the weakest individual has the smallest share of the roulette wheel. To select an individual, a random number is generated from a uniform distribution in the interval [0, the sum of the N fitness values] and the individuals are chosen based on where the pointer stops. This process is repeated until the desired number of individuals has been selected. It was assumed that 90% of the fittest parents will remain in the future generation.

1) The numbers in each area represent the number of chromosome (i.e., the number of policy) and the fitness (i.e., total net benefit)

Figure ‎6.. Roulette wheel selection operator applied to the GA
After the parents were selected on a fitness basis, two or more of the parent chromosomes were recombined to produce new individuals. This model used the double-point crossover operator, which exchanges the bits in both parents where two crossover points are selected randomly (see Figure ‎6.). The probability that crossover happens at a particular couple (i.e., crossover rate) was assumed to be 70%.

Figure ‎6.. Double-point crossover operator applied to the GA

Mutation was performed on a bit-by-bit basis where one or more bits chosen were flipped from 0 to 1 or vice versa with the probability of 10% for an individual (see Figure ‎6.). The infeasible offspring was rejected, whereas the feasible offspring was evaluated and reinserted into the general population.

Figure ‎6.. Mutation operator applied to the GA

The GA was terminated after repeating for the number of generations selected. The population-based GA itself involves parallel processing (i.e., the individuals in the current population are evaluated at the same time). In addition to the parallelisation of fitness evaluations, the hypertension SDDP model also allowed parallel computation in 100 replications in calculating the objective function through the underlying evaluation model.

The GA toolbox developed in the department of Automatic Control and Systems Engineering at the University of Sheffield^[367] was modified for the hypertension SDDP model. Modifications were mainly made to generate the initial population and the feasible individuals in the next generations in the context of the hypertension SDDP. The pseudo-code of the applied GA is presented in Figure ‎6..

% Select the parameters for genetic algorithm. nGeneration = 50; % The number of generations. nInitPop = 30; % The size of initial population. % Select an initial population randomly. InitPop = randi(4128, nInitPop)); mChrom = dec2bin(InitPop); % Encoded InitPop into binary vectors. mPhen = bin2dec(mChrom); % Convert chromosomes into real values. % Evaluate the initial population via the underlying evaluation model. This provides the set of total net benefits of the policies included in the initial population. [ObjVal(:)] = EvModel(mPhen(:)) % Repeat for the number of generations selected. WHILE iGeneration < nGeneration % Assign fitness values to the population based on the total net benefits. vFitVal = ranking(ObjVal(:,:)); % Select individuals for breeding using the roulette wheel selection 'rws'. 90% of the fittest patents are selected for bleeding. [mSelChrom] = select('rws',mChrom,vFitVal 0.9); % Recombine the selected individuals using the double-point 'xovdp'. The probability of crossover at a particular parent is 70%. [mSelChrom] = recombin('xovsh',mSelChrom,0.7); % Perform mutation on the offspring with the probability of 10% for an individual. [mSelChrom] = mut(mSelChrom,0.1); % Convert the offspring chromosomes into real values and then evaluate them via the underlying evaluation model. mSelPhen = bin2dec(mSelChrom); [mSelObjVal(:)] = EvModel(mSelPhen(:)); % Reinsert offspring into the general population. [mChrom,mObjVal] = reins(mChrom,mSelChrom,mObjVal,mSelObjVal); % Save the best solution and the maximum reward in each generation. [MaxV(iGeneration),MaxIdx(iGeneration)] = max(mObjVal); % Update the generation number. iGeneration = iGeneration+1; END % Decide the optimal solution and the reward where the optimal solution was used. OptSol = Policy(MaxIdx(end)); OptV = MaxV(end);

Figure ‎6.. Pseudo-code of the GA used for the hypertension SDDP model