The guiding principle in this monograph is to develop a new theory of modular forms which encompasses most of the available theory of modular forms in the literature, such as those for congruence groups, Siegel and Hilbert modular forms, many types of automorphic forms on Hermitian symmetric domains, Calabi-Yau modular forms, with its examples such as Yukawa couplings and topological string partition functions, and even go beyond all these cases. Its main ingredient is the so-called 'Gauss-Manin connection in disguise'.