There's not much to do to accelerate on empty plot (however i wouldn't consider 6ms to be a lot of time, i mean that's probably unperceivable for humans)
However there's different things you can use to accelerate plotting in general.
1. Use openGL accelerator (setOpenGL(true) in your qcustomplot), if you have a gpu this will use the gpu computing power instead of just the cpu.
2. When displaying different graphs use adaptive sampling (setAdaptiveSampling(true) on the QCPgraphs that you add to your plot), this is an algorithm that reduces the number of points used to display your graph but also keeps a good visualization (reducing computing time)
3. Read the following page https://www.qcustomplot.com/documentation/performanceimprovement.html where derManu described different things about plot performance
How quick do you need to go ?