There are probably as many stories about the Olympic Village as there are athletes who stay there. The accommodations, which house the competitors at the Summer and Winter Games, include a bank, a salon, a post office, a massive cafeteria and — yes — a McDonald’s.

It’s part dormitory, part dining hall, part lounge; a place to prepare for the defining moment in any athlete’s life and a place to unwind or people-watch before or between competitions.

Despite its sprawl, the village remains off-limits to basically everyone in the world, making it the persistent object of fascination from Olympic spectators every two years. What really happens when you take a few thousand of the world’s top competitors and put them all in the same space?

PEOPLE spoke with some gold medalists to find out. Their comments, together with those of others Olympians in previous interviews, provide an enlightening look inside.

‘It’s Like a Giant College Campus’

Ask swimmer Natalie Coughlin about the biggest misconception people might have about the Olympic Village and she’ll tell you: “I think they think it’s a lot more glamorous than it is. It’s really pretty boring.”

“It’s like a giant college campus — except that everyone is getting ready for the biggest event of their life so they’re incredibly stressed,” the three-time Olympian and 12-time medalist, including three golds, tells PEOPLE.

“They’re really, really focused and so everyone is 100 percent on their best behavior up until the time that they’re done competing,” Coughlin says. (Other athletes have had … wilder experiences. More on that below.)

“When people are done competing, they generally leave the Olympic Village for the sake of the athletes who are still competing,” she says. “Because you want to enjoy all the fun parties and really enjoy the Olympics, but you do not want to take away from the athletes who are still competing.”

Runner Allyson Felix — a track and field sensation in four Olympics between 2004 and 2016, with six gold medals — says something similar.

“Everyone always assumes it’s like this big party and this really crazy place, and I think the real thing is it’s just a mix of a lot of different people,” she tells PEOPLE. For example, she says, there will be athletes whose real victory was earning an Olympic berth in the first place, so they look to soak up more of the atmosphere of the Games.

For others, however, being on the podium is the only thing on their mind.

The Rooms Aren’t as Fancy as You Might Think

It’s best to imagine the athletes’ accommodations as a large network of college dorm rooms or suites, Felix says. And, just like most dorms, the athletes’ rooms are “very spartan,” according to Coughlin. The beds pose a particular challenge for the taller athletes — think Nathan Adrian or Matthew Grevers — who get around the issue with extenders.

“Here you are at the hardest competition of your life and you have a twin bed,” Felix says.

Dorm rules apply in other ways, too, she and Coughlin say: The athletes — from different age groups and backgrounds — share rooms, common areas and bathrooms. They are grouped by sport, taking up blocks or floors of rooms. Some of the Olympians are in their teens, others in their 40s.

“You might have someone who’s just out of high school and you might have someone who has a whole family,” Felix says. For a few weeks, they’re roommates.

RELATED VIDEO: Olympic Hopeful Figure Skaters on How to End a ‘Perfect Performance’ in 5 Steps

At Coughlin’s first two Games, in Athens and Beijing, she says their bathroom didn’t even have a bathtub, creating a bit of a hazard for the female swimmers all staying together (and who had shave regularly).

She says she learned to bring a lot of photos with her from home — plus some painter’s tape — to add personality to her space. Otherwise, with all the white walls, “it can seem like a hospital room.”

There are two Olympic Villages at the South Korean Games this month, one at each of the hubs of activity on the coast and in the mountains, which together will host more than 6,500 athletes and team officials.

Yes, They Do All Run Into One Another All the Time

Coughlin says the best spot to be in the village is the cafeteria. Not only does it provide a place to intermingle with the rest of the athletes from your country, it’s a great opportunity for people-watching. One of Coughlin’s favorite games? Guessing a person’s sport based on their body type. (The basketball players usually give themselves away.)

She says the cafeteria is “gigantic” and likens it, in terms of scale, to a wholesaler like Costco. With few exceptions, it’s the only grub game available.

“If you go at peak times, it’s maybe one of the craziest experiences I’ve ever had,” rugby player Jessica Javelet told the Washington Post in 2016.

“We’d graze over our food for hours watching all the eye candy, wondering why I got married,” soccer gold medalist Julie Foudy told ESPN in 2012.

The food on offer spans the world, Coughlin says. When it comes time to finding a seat, high-school logic kicks in: Over time, the same athletes will sit at the same tables.

During downtime, the athletes can gather in lounges to watch other competitions, to call home or just to hang out and play a game. In Beijing, Coughlin remembers there being a lounge pool for the swimmers and other athletes — they called it the “discotheque pool.” In Rio in 2016, one such lounge was stocked with “free condoms, Dance Dance Revolution, and all the limited edition athlete-only golden Coca-Cola bottles you can get your hands on,” Greek runner Alexi Pappas told The New York Times.

Given all this proximity, it’s inevitable for athletes of all kinds to cross paths.

Coughlin shares this story: While in London for the 2012 Games, at one point applause broke out at the arrival of a group of people. Coughlin assumed it was one of the British royals — “and it ended up being Usain Bolt and his hype man.”

Felix has also had her run-ins: In 2008, she came across the entire U.S. men’s basketball team, featuring Carmelo Anthony, Kobe Bryant, and Dwyane Wade. In Rio eight years later, the ladies’ gymnastics team was staying a few floors away.

“It’s always funny because the village is such a mix of people,” Felix says.

Sometimes this even sparks a romantic connection: Cyclist Taylor Phinney told ESPN in 2012 how he wooed gymnast Shawn Johnson in Beijing in part by throwing her up candy bars from the balcony of his room one floor below.

About That ‘Extremely Long Line’ at McDonald’s

There are a variety of explanations about McDonald’s enduring popularity among athletes at the Olympics, but they basically all boil down to this: Fast food is comfort food, it tastes like America, it’s readily available and it’s free (within reason).

According to local reports, there will be multiple McDonald’s for the South Korean Games: one for visitors (which will be shaped like a burger, fries and soda) and one for athletes. However, according to the Washington Post, McDonald’s’ presence is smaller now than in years past — although they will still be handing out free Big Macs in the Olympic Village.

“It’s one one of the most popular stops at the village, always an extremely long line, and I think it’s just about what you know,” Felix says. “Sometimes when you’re just across the world, you want something that you know exactly what it’s going to taste like.”

Even Coughlin, who notes that she’s not a regular fast-food person, couldn’t resist a regular Egg McMuffin. And it’s not just the Americans: When TIME’s Sean Gregory surveyed the members of a typically winding line at the Rio location in 2016, he got a range of responses from athletes from Malta, Montenegro and elsewhere.

“This is like confessing my guilt to a priest,” Maltese swimmer Nicola Muscat told him of admitting to indulging.

Egypt’s Mohad Ishak was perhaps most direct, telling Gregory: “The food in the dining hall is not good. And I have to eat something.”

‘There’s a Lot of Sex’

It may be the stuff of gossip and titillation, but the fact remains that many in the Olympic Village use the close-quarters as a chance to get very personal with some of their other fellow competitors.

Many of those athletes spoke at length to ESPN about their excursions for a 2012 piece. Among the anecdotes recounted was a days-long gathering during the 2000 Summer Olympics, in Sydney, complete with “an Oakley duffel bag overflowing with condoms.”

“There’s a lot of sex going on,” two-time gold medaling soccer player Hope Solo told ESPN. Swimmer Ryan Lochte, who has six gold medals over four Olympics, put the exact amount at “70 percent to 75 percent of Olympians.”

“It’s like the first day of college,” water polo player Tony Azevedo said of those first few days all together in the athlete village. “You’re nervous, super excited. Everyone’s meeting people and trying to hook up with someone.”

It’s “a pretty wild scene, the biggest melting pot you’ve been in,” said swimmer Eric Shanteau. To accommodate, officials provide tens of thousands of condoms, such as 450,000 condoms for the Rio Games, USA Today reported.

Skater Adam Rippon has since joked about the “generic” condoms being given away in South Korea.

Adam Rippon (@Adaripp) ranting for a minute straight on Instagram about the #Olympics condoms is honestly a MOOD. pic.twitter.com/y2JKlpLT3c

— Bo Bayerl (@BoBayerl) February 10, 2018

https://platform.twitter.com/widgets.js

“Everyone talks about how there are so many condoms for each athlete, but they don’t just give them to you,” Swiss swimmer Alexandre Haldemann told the paper. “You have to go and get them yourself, and you can take as many as you want.”

Felix tells PEOPLE the level of shenanigans, such as they are, varies depending on the athletes involved and if they’ve already competed or not or what their priorities are. “You can kind of find everything,” she says.

According to Coughlin, the rule is usually that after an athlete finishes competing, it’s best to vacate the village so that they can party and stay out late without disturbing others.

What all of this adds up to — romps and romance or not — is an experience unlike any other, and one that is inaccessible to all but a select few.

“You get to meet people from all over the world that typically you wouldn’t even come into contact with,” Felix says. “To me that’s one of the coolest things … Being with people from other countries, all having to be together, is a really special experience.”

The 2018 Winter Olympics are airing live on NBC. To learn more, visit teamusa.org.