A twenty-page document can take up forty megabytes or two hundred kilobytes with the same visible content. The difference is almost never the text.
Text weighs nothing; images do
A full page of text is a few thousand characters, and each character takes up very little: the text of an entire book fits comfortably in the space of a single phone photo. What fills a PDF is the embedded images, and it is worth knowing that a screenshot pasted into a document is stored at full resolution even when it appears on the page the size of a stamp.
Scanned versus exported
This is the split that explains almost every case. A PDF exported from a word processor contains real text: letters the computer understands as letters. A scanned PDF contains no text at all, but a photograph of each page. That is why a ten-page contract that was scanned can weigh fifty times more than the same contract exported, and why you can search for a word in one and not in the other.
How much room compression has
It depends which of the two you have. In an exported PDF there is a lossless rewrite that reorders the document and trims what is redundant without touching a pixel or breaking the text; it usually gains a good deal and costs nothing. In a scan that route barely pays, because the weight is in the images, and to come down meaningfully they have to be re-compressed. When even that is not enough, the only way out is turning the pages into lower-quality images, which loses the selectable text: a trade worth making knowingly.
What to do before compressing
If you generated the PDF yourself, look at the source first. Shrinking images before you put them into the document beats compressing the finished PDF, because you avoid storing detail only to throw it away again. If the document arrives already made, compressing is the tool you have, and it pays to start with a generous target: every extra megabyte you take off is paid for in quality.