Sequential drug decision problems in long-term medical conditions: a case Study of Primary Hypertension Eunju Kim ba, ma, msc

Table of contents

Abstract 4

Table of contents 6

List of tables 7

List of figures 8

Abbreviations 9

Chapter 1.Introduction 11

1.1Chapter overview 11

1.2Introduction to sequential drug decision problems 11

1.3General optimisation approaches to solve sequential decision problems 16

1.4Motivations and research objectives 20

1.5Thesis structure 21

Chapter 2.Problem specification for sequential drug decision problems 22

2.1Chapter overview 22

2.2Classification of evaluation models for sequential drug decision problems 22

2.3Comparison between sequential drug decision problems and representative combinatorial optimisation problems 27

2.4Mathematical description of sequential drug decision problems 31

2.5Computational complexity of sequential drug decision problems 34

Chapter 3.Potential optimisation methods for sequential drug decision problems 37

3.1Chapter overview 37

3.2Systematic review on approximate optimisation methods 37

3.3Simulated annealing 52

3.4Genetic algorithm 54

3.5Reinforcement learning 55

3.6Application of the key optimisation methods on a hypothetical case 64

Chapter 4.Modelling sequential drug decision problem for hypertension: Overview 79

4.1Chapter overview 79

4.2Hypertension and pharmacologic management 79

4.3Previous economic evaluations in primary hypertension 87

4.4Conceptual framework of the hypertension sequential drug decision problem 89

4.5Implication for the hypertension SDDP modelling 99

Chapter 5.Modelling sequential drug decision problem for primary hypertension: evaluation model 101

5.1Chapter overview 101

5.2Population 101

5.3Time 102

5.4The structure of the short-term drug switching model 103

5.5The structure of the long-term CVD model 108

5.6Treatment effectiveness and costs 112

Chapter 6.Modelling sequential drug decision problem for hypertension: Implementation 133

6.1Chapter overview 133

6.2Overview of the hypertension SDDP optimisation model 133

6.3Enumeration 144

6.4Simulated annealing 154

6.5Genetic algorithm 156

6.6Reinforcement learning: Q-learning 161

Chapter 7.Modelling sequential drug decision problem for hypertension: Results 169

7.1Chapter overview 169

7.2Model validity 169

7.3Markov model-based optimisation: Enumeration 178

7.4Simulated annealing 200

7.5Genetic algorithm 205

7.6Reinforcement learning: Q-learning 212

7.7Summary of the case study of primary hypertension 217

Chapter 8.Discussion 223

8.1Chapter overview 223

8.2Summary of research 223

8.3Future research 232

Chapter 9.Conclusion 238

Reference 243

Appendix 1. Exploratory literature review on previous studies, which evaluated the cost-effectiveness of sequential treatment policies for long-term medical conditions 278

Appendix 2. The target papers for the systematic review on approximate optimisation methods 288

Appendix 3. Full search strategies of the systematic review on approximate optimisation methods 289

Appendix 4. The parameters used to populate the hypothetical SDDP model 294

Appendix 5. Summary of the included studies in the literature review on previous economic evaluations in primary hypertension 296

Appendix 6. Pooled standard deviation of SBP lowering effect 300

Appendix 7. The list of 4,128 treatment sequences tested in the hypertension SDDP model 307

Appendix 8. Simulated annealing results depending on the size of the neighbourhood 367