Honestly, it's just the hassle of tourist areas in general. There is residential not too far off but once you enter the tourist area, traffic gets worse and you're dealing with non-local drivers who don't know where they're going, super-wide busy roads with lots more busses and charters, etc. Prices for things like simple fast food and gas go up - you're in the tourist bubble so everything is tourist centered.
If I were new to the area or in my very late teens or early twenties, I might find all of this exciting but since I'm not, it's just kind of a pain in the booty.
Security is a small part of it but getting a spot to park (the garages have helped with this but during busy times, it's still a bit of a hassle) and then getting spit out in one of the few entry points from garages and having to huff it to whatever store you're after in whatever the Florida weather decides to be that day - usually somewhere between humid to the point that you're dripping wet to humid to the point that the sky is doing the dripping or both within 20 minutes of each other. (it's always fun to be on the other side of DS from your car when the rain starts and you think about the umbrella you left safely locked away and dry on the covered third story of a garage)
For me personally, there are places like UNIQLO that I like and which aren't always the easiest to shop online, with only a handful of locations in the US that makes it pretty unique but now even they have a spot in Florida Mall.
Some people look down on FM which also has it's share of tourists as well as general slice-of-life (read into that what you will) locals but it is a simpler affair to get in and out of and again, for the people that like to quibble over atmosphere, I have to ask if they are there to shop or to people-watch. They aren't mutually exclusive things and I don't mean to talk about tourists as if they are second-class citizens or people I don't like but for just normal every-day stuff, when you've got places to go and things to do, it's easier to not be trying to get past people who aren't in that time and place, sharing similar concerns. (I'm fine being one of them in the parks as long as we're not about to lose our fastpass window on something, btw.)
There are a few places that have sort of unique non-Disney, non-touristy things (think not Coke or soon-to-be M&M) but for the most part, I wouldn't say unique enough to be worth the hassle and the markup (sometimes a lot, sometimes a little) for me. If I were a runner, maybe there would be something unique and special in that store that would draw me there that I either couldn't get somewhere else local like one of the big sporting stores or wouldn't want to order online without seeing/feeling but I'm not, so I really can't say.
It doesn't seem like they are that particuarly special to me but that's how it is with every niche thing for people from the outside looking in, I guess, so I could be wildly off the mark with my opinion, there.
If I didn't have an annual pass and wanted a cheap Disney "fix" I could see heading over there for a bit. I guess if I liked to drink and gawk at tourists, that might be a thing for me, too but (and I don't claim to speak for all locals) honestly, we (my family/friends) go there sometimes for some of the restaurants when we are making a specific trip with that in mind and may wander around a bit or when we have guests from out of town who are or aren't going to the parks.*
If you live near anyplace that has tourist draws (think major cities, a coast with beaches, etc.) how often do you really want to venture into the tourist areas if you don't have to for casual stuff?
I have friends who live in Vegas and they do just about everything they can to avoid the strip and old downtown about 98% of the time for the same reasons.
I don't think it's a Disney or Disney Springs specific thing.
Reading this, it makes it sound like I might not like WDW in general but honestly, having lived in FL almost my whole life, for me it's a local theme park. I see WDW (perhaps through spoiled lenses) the way other people might see Kennywood or a Six Flags so there is a little bit of a logical disconnect in my mind, there... Having a young child, that's of course, a thing, too. :)
*also for a show every now and then at House of Blues, again, when the world isn't ending.