Abstract: Nonlinear dynamical systems with an irrational restoring force often occur in both science and engineering, and always lead to a barrier for conventional nonlinear techniques. In this paper, we have investigated the global bifurcations and the chaos directly for a nonlinear system with irrational nonlinearity avoiding the conventional Taylor's expansion to retain the natural characteristics of the system. A series of transformations are proposed to convert the homoclinic orbits of the unperturbed system to the heteroclinic orbits in the new coordinate, which can be transformed back to the analytical expressions of the homoclinic orbits. Melnikov's method is employed to obtain the criteria for chaotic motion, which implies that the existence of homoclinic orbits to chaos arose from the breaking of homoclinic orbits under the perturbation of damping and external forcing. The efficiency of the criteria for chaotic motion obtained in this paper is verified via bifurcation diagrams, Lyapunov exponents, and numerical simulations. It is worthwhile noting that our study is an attempt to make a step toward the solution of the problem proposed by Cao Q J et al. (Cao Q J, Wiercigroch M, Pavlovskaia E E, Thompson J M T and Grebogi C 2008 Phil. Trans. R. Soc. A 366 635).

Key words: nonlinear dynamical system, Melnikov boundary, irrational restoring force, saddle-like singularity, homoclinic-like orbit