You could click a button to clear the list or use other history-wiping tools to erase it. In earlier versions of Safari, the Downloads button was always present and all previous downloads appeared in a list. While it seems like a simple matter of clicking a button and scrolling, Apple added some sophistication that has tripped people up.
A few readers have asked about the downloads list in macOS Safari-in particular, how to find it when the button is missing and where to find entries for older downloads.