We investigate quantum point contacts (QPCs) fabricated from GaAs two-dimensional hole gases. We show that several surprising aspects of magnetic focusing experiments can be understood form the presence of a crossing point at finite momentum of the lowest two spin subbands. The crossing point originates in one-dimension from a strong two-dimensional Rashba spin-orbit interaction of holes in asymmetric quantum wells, with a non-linear (cubic) dependence on momentum. We also discuss how a magnetic field parallel to the channel, or an asymmetry in the QPC lateral potential, can remove the degeneracy at the crossing point. These features allow us to explain the anomalous sign of the spin polarization filtered by the QPC, as well as a surprising non-monotonic dependence of spin-polarization on magnetic field. Controlling the magnitude of the spin-splitting affords a novel mechanism for inverting the sign of the spin polarization.