Information Measure as Time Complexity

Abstract

The concept of Shannon entropy of random variables was generalized to measurable functions in general, and to simple functions with finite values in particular. It is shown that the information measure of a function is related to the time complexity of search problems concerning the functions in question. Formally, given a Turing reduction from a search problem f(x)=y to another function g(x), the amount of information about f(x)=y provided by querying g(x) is exactly equal to the average mutual information I(f;g). As a result, the average number of queries is I(f=y)/I(f;g), where I(f) is amount of self-information about the event {f=y}. In the idea case, if I(f=y)/I(f;g) is polynomial in the size of input and the function g(x) can be computed in polynomial time, then the problem f(x)=y also has polynomial-time algorithm. As it turns out, our information-based complexity estimation is a natural setting in which to study the power of randomized or probabilistic algorithms. Applying to decision problems, our result provides a strong evidence that P=RP=BPP. Using brute-force search as a benchmark for efficiency, our results also support that P!=PP. Further, applying an argument similar to Carnot's work on measuring the efficiency of an ideal heat engine in thermodynamics, it is shown that I(f=y)/H(f) is a lower bound on query complexity of solving f(x)=y. According to Markov's inequality, if I(f=y)/H(f) is exponential in the size of input, then the probability of f(x)=y having polynomial-time algorithm is a negative exponential in the size of input. This result enables us to reduce the problem of proving FP!=FNP to the computation of the entropy of certain simple functions, such as the subset sum function. As a result, P!=NP is provable, answering a long-standing open problem.