When wireless decision-making entities have complete information, the network control problems are frequently addressed in the model-based paradigm. However, due to the practical limitation of information incompleteness/locality, directly applying the model-based solutions will face difficulties. As a result, the method of controlling-by-learning without the need for the a priori network model, namely, the model-free learning, has been considered as one promising implementation approach to wireless networks. In this thesis, the applications of the model-free learning in three networks with different characteristics and objectives are investigated, they are an opportunistic spectrum access (OSA) network, a cloudlet-based mobile cloud computing (MCC) network, and an energy harvesting based network. Effective and efficient model-free learning mechanisms in these networks are aimed to be designed. Both analytical and numerical results validate the efficacy of the proposed algorithms.