Who decides which HS code applies to my goods?

The importing country's customs authority, not the supplier and not the freight forwarder. The supplier can propose a code and the forwarder can advise, but the legal classification is determined at the destination, and the importer of record carries the liability for it. This is why a "the factory always ships it as code X" explanation is not protection: if the code is wrong, the duty assessment, the penalty and the seizure risk fall on the importer.