MDP-Based Network Selection with Reward Optimization in HetNets

We investigate the network selection problem over heterogeneous networks. We formulate the decision making problem as a Markov decision process (MDP). Two Poisson processes are introduced to model user arrival and departure, and the change of the number of user in a specific network is described by Skellam Distribution. We apply the Total order preference by similarity to the ideal solution (TOPSIS) method to address the issue of state space explosion in MDP model. To acquire the optimal network selection scheme, the value iteration algorithm is employed to solve the MDP model. We take simulations in Matlab to prove the effectiveness of the proposed scheme. From the experiment results, it is found that our proposed scheme can reduce handover probability by 7.5% and 40.4% at the early stage of connection, compared with the Infinite MDP-based (IMDP) strategy and TOPSISonly strategy. It can also reduce blocking probability by 6.1% compared with IMDP, while maintaining desirable throughput for the target user.