Lactose-free milk still makes my stomach gurgle.

Hello. I've stopped tolerating lactose a few years ago and cut out milk and any milk-related products which aren't fermented in some way (these seem to not affect me as much, sometimes at all). I tried replacing normal milk with lactose-free substitute but my symptoms are just as bad after consuming it(feeling bloated, diarrhea, etc.). Could it mean that I'm just allergic to milk and not actually lactose intolerant? Just a cup of flat white is enough to make my day bad.