Scheduling complex production processes in real time is a challenging task because it typically takes hours to find optimal schedules. In recent years, reinforcement learning (RL) has shown great potential for solving complex scheduling problems. An appropriately trained RL agent can quickly respond to similar situations with near-optimal strategies to achieve good enough or even brilliant performance. This work presents an efficient methodology to apply the deep Q-learning algorithm to integrated process planning and scheduling. The presented RL methods were proven to be efficient in finding near-optimal schedules in real time. Meanwhile, the trained RL agents show great flexibility in handling process deviations without sacrificing production performance.