In this paper, a novel hybrid harmony search (HHS) algorithm based on the integrated approach, is proposed for solving the flexible job shop scheduling problem (FJSP) with the criterion to minimize makespan. First of all, to make the harmony search (HS) algorithm adaptive to the FJSP, the converting techniques are developed to convert the continuous harmony vector to a kind of discrete two-vector code for the FJSP. Secondly, the harmony vector is mapped into a feasible active schedule through effectively decoding the transformed two-vector code, which could largely reduce the search space. Thirdly, a resultful initialization scheme combining heuristic and random strategies is introduced to make the initial harmony memory (HM) occur with certain quality and diversity. Furthermore, a local search procedure is embedded in the HS algorithm to enhance the local exploitation ability, whereas HS is employed to perform exploration by evolving harmony vectors in the HM. To speed up the local search process, the improved neighborhood structure based on common critical operations is presented in detail. Empirical results on various benchmark instances validate the effectiveness and efficiency of our proposed algorithm. Our work also indicates that a well designed HS-based method is a competitive alternative for addressing the FJSP.