Most of the time you should be watching the ball, but not always. If the ball is moving slowly, you should watch it all the way into your racket. If the ball is moving fast, you can't react at the last second, so there's no point in watching the ball right to contact. In both cases, as soon as watching the ball no longer is beneficial, you should be looking up to watch your opponent. You should generally be aware of the opponent and what he's doing and where with peripheral vision, but actually looking up to watch him is even better. That allows you to quickly prepare for whatever he's doing. You should pick up the ball again just before he hits it, and then watch the ball the rest of the way - until it's time to look up again to see your opponent.