In wireless networks, network state information (NSI) usually consists of channel state information (CSI) and queuing state information (QSI). NSI, especially CSI, has been widely used in designs and conﬁgurations of wireless networks. However, most of the existing works assume that NSI is perfectly available at the decision making entities in the network. In general such assumption is not practical because of various limitations to acquire perfect NSI. Especially, in the context of the emerging wireless networks considered in this dissertation, it is crucial to consider imperfect NSI because
it is challenging to measure and to convey perfect NSI in these systems. Due to the inaccuracy of NSI, it is challenging to make optimal decisions.
In this dissertation, we address those issues under the framework of stochastic optimizations. We ﬁrst consider coordinated multi-point cellular networks with delayed CSI. The base station clustering and rate allocation problem in uplink is formulated as a networked Markov decision process, for which we derive the optimal policy with low computation cost. We further study how to provide better support for mobile cloud computing services in a
cloud radio access network (C-RAN) in the second wireless networking paradigm. We formulate the problem by maximizing the system throughput while constraining the user response latency within speciﬁed values. The third wireless networking problem discussed is the resource sharing problem for software-deﬁne device-to-device communications in virtual wireless networks given imperfect NSI. The problem is formulated as a discrete stochastic optimization problem addressed by the proposed discrete stochastic approximation algorithms. The last type of wireless networks considered is unmanned aerial
vehicle (UAV) ad hoc networks, where CSI measurements suﬀer from the high mobility of nodes and the challenging tactical environment. Discrete stochastic approximation based algorithms are also developed to combat the challenging operation environment of UAV ad hoc networks.
With the tools from stochastic optimizations, we can reduce the eﬀect of imperfect NSI in those wireless networks. Extensive computer simulations are presented to show that our proposed schemes can outperform the existing schemes.