It’s not true that Bitcoin’s value is set solely by speculators. It has a use value which would exist even if all the speculators vanished. I know a company that pays their employees in Bitcoin – not because they think its value will go up, but because it’s cheaper than the transaction costs involved in fiat-denominated payments. I know a lady in Zimbabwe who is using Bitcoin because she does not have access to a sound currency.
These people are not speculators: they may prefer to trade their Bitcoin for a fiat currency at the first chance. However, their ongoing usage provides a demand for Bitcoin and thus establishes a price floor. The price of Bitcoin is the combination of current non-speculative usage and expectations for future non-speculative use cases for the Bitcoin network.
This is different than a purely speculative asset like rare stamps or baseball cards, which have no current or future use value.