TCP is not a packet delivery system, it is a stream delivery system. There is no separate packets. Just a stream of bytes that is added to once in a while.
Your system is flawed from the beginning since it assumes something that outright isn't true. TCP may cause packets to be sent, yes. But all the application sees is a pair of byte streams with no special separation marks.
It is fully legal for multiple "packets" to be joined like this. Or it could hold sending half of one "packet" for 5 sec while letting the rest be sent instantly. Your code has to deal with the fact that it's a bytestream and not packets.