The existence of merging black hole-neutron star (BHNS) binaries has been ascertained through the observation of their gravitational wave (GW) signals. However, to date, no definitive electromagnetic (EM) emission has been confidently associated with these mergers. Such an association could help unravel crucial information on these systems, for example, their BH spin distribution, the equation of state (EoS) of NS and the rate of heavy element production. We model the multi-messenger (MM) emission from BHNS mergers detectable during the fourth (O4) and fifth (O5) observing runs of the LIGO-Virgo-KAGRA (LVK) GW detector network, in order to provide detailed predictions that can help enhance the effectiveness of observational efforts and extract the highest possible scientific information from such remarkable events. Our methodology is based on a population synthesis-approach, which includes the modelling of the signal-to-noise ratio of the GW signal in the detectors, the GW-inferred sky localization of the source, the kilonova (KN) optical and near-infrared light curves, the relativistic jet gamma-ray burst (GRB) prompt emission peak photon flux, and the GRB afterglow light curves in the radio, optical and X-ray bands. The resulting prospects for BHNS MM detections during O4 are not promising, with an LVK GW detection rate of 15.0+15.4-8.8 yr−1, but joint MM rates of ∼10−1 yr−1 for the KN and ∼10−2 yr−1 for the jet-related emission. In O5 we find an overall increase in expected detection rates by around an order of magnitude, owing to both the enhanced sensitivity of the GW detector network, and the coming online of future EM facilities. Considering variations in the NS EoS and BH spin distribution, we find that the detection rates can increase further by up to a factor of several. Finally, we discuss direct searches for the GRB radio afterglow with large-field-of-view instruments during O5 and beyond as a new possible follow-up strategy in the context of ever-dimming prospects for KN detection due to the recession of the GW horizon.