As fundamental parameters of the Sun, the Alfv\’en radius and angular momentum loss determine how the solar wind changes from sub-Alfv\’enic to super-Alfv\’enic and how the Sun spins down. We present an approach to determining the solar wind angular momentum flux based on observations from Parker Solar Probe (PSP). A flux of about $0.15\times10^{30}$ dyn cm sr$^{-1}$ near the ecliptic plane and 0.7:1 partition of that flux between the particles and magnetic field are obtained by averaging data from the first four encounters within 0.3 au from the Sun. The angular momentum flux and its particle component decrease with the solar wind speed, while the flux in the field is remarkably constant. A speed dependence in the Alfv\’en radius is also observed, which suggests a “rugged” Alfv\’en surface around the Sun. Substantial diving below the Alfv\’en surface seems plausible only for relatively slow solar wind given the orbital design of PSP. Uncertainties are evaluated based on the acceleration profiles of the same solar wind streams observed at PSP and a radially aligned spacecraft near 1 au. We illustrate that the “angular momentum paradox” raised by R\’eville et al. can be removed by taking into account the contribution of the alpha particles. The large proton transverse velocity observed by PSP is perhaps inherent in the solar wind acceleration process, where an opposite transverse velocity is produced for the alphas with the angular momentum conserved. Preliminary analysis of some recovered alpha parameters tends to agree with the results.