You found the right place. For anyone landing here later: computer authorizations are managed from a computer or phone, not from the Apple TV itself. In the Apple Music or Apple TV app on a Mac or PC, Account > View My Account > Computer Authorizations has Deauthorize All, and it also exists under Media & Purchases in Settings on an iPhone or iPad. Two catches: Deauthorize All only appears when you have two or more computers authorized, and it can only be used once every 12 months. After using it, reauthorize the computers you still own from the Account menu. It does not affect the devices list for downloads, that is managed separately under your account's devices.