A lot of villains, unfortunately, have to be just smart enough that the protagonist can outsmart them. The authors of villains like this don't want them to outpace the rather slow, drawn-out mystery plot. I suspect that much would be improved by letting villains be intelligent in the ways you suggest, and also allowing protagonists be intelligent enough to counter such villains.
Good point. Although maybe we should be somewhat more sympathetic to the authors - it probably is hard to write really intelligent characters, particularly characters that are smarter than they are, without it coming off as the plot bending to their whims.
I think it is hard. On the other hand, I think a lot of time people write characters they think are intelligent, when really they're no such thing. It would be easier to write intelligent people if they let them make some mistakes and wrong guesses and gradually get closer to the truth, instead of just putting everything together at once or in some great blaze of revelation.