Minecraft, 100%. You can set it to peaceful mode so no enemies spawn, and even mess with the world settings so more structures generate in your map.

The putt putt line of games. They run on scummvm and my kids love them. Later kings quest, and stardew valley. The lego marvel super heroes on Xbox 360 (I think there’s a pc version) has an entire Manhattan island that they can roam around freely and interact with.

Putt-putt, pajama sam a.d freddy fish are great!

Slime Rancher might be worth looking into.

There’s a few short indie adventure games that may fit the bill:

• The First Tree
• Spirit of the North
• Abzu
• Aer
• Proteus
• Untitled Goose Game (Not an adventure game, but good goofy fun)
• Donut County (Not adventure, but a very simple and accessible puzzle that spans an entire town)

I haven’t played Stray, but it may be a good fit. I also haven’t played past the opening scene of Firewatch, but if your daughter can manage walking around Skyrim then I think it should be okay.

I searched for indie exploration games. City of Muse came up.

There’s a list of 3D exploration games on Itch.

The Peppa Pig game is surprisingly open. If you just walk off screen it lets you keep going and you end up in new locations. Hop on the bus and end up in another location. There aren’t many blocks to stop you from going where you feel like going and there’s a variety of activities at each location.

These aren’t exactly exploration games, but they’re simple games that my toddler likes too:

• Animal Crossing is easily her favorite. She loves “helping” my wife pick outfits and fish.
• A Building Full of Cats is short, cheap, and cute. She likes making up stories about each apartment and cat. There’s also tons of similar games in different locations.
• Cats in Time has simple puzzles that she can do with a bit of help.
• Slime Rancher might be a good fit. It’s simple and cute with a focus on exploration.
• Dorf Romantik is a relaxing and cute game that’s a good introduction to resource management. She might not be good at the actual goal of the game, but she likes placing tiles.
• Subnautica in creative mode might be interesting for exploration, depending on how sensitive your kid is about some of the darker areas and creatures.

If you’re open to dealing with emulation, both the new Zeldas pretty much fit the bill. There’s combat but probably less than Skyrim.

Slime Rancher is one I enjoyed for a while that’s definitely kid friendly. Supraland didn’t really grab me, but in terms of being super tame and having varied stuff to explore it fits again.

If it specifically has to be houses/cities, none of those fit that well. But they have worlds that are varied and interesting.

Maybe Stardew valley would scratch that itch. Or some old rpgs like FF or chrono trigger.

Outer Wilds

Might be a little advanced for a 4yo, but it definitely checks the exploration/adventure mark.

Since you’re on PC, using cheats or potentially mods can expand the list of games you have available, since you might be able to just ignore certain aspects of a game, like combat, money or resources in general.

The more recent Assassin’s Creed games have something called a Discovery Tour, where you can just explore the world, without combat. They are partially meant for educational purposes and can teach about the different regions and history, and offer guided tours. I don’t know if there’s a real free roam, where you can just do whatever though.

You mentioned buying Goat Simulator, so maybe other of these “Simulator” games might be for you.

I don’t know how much you can go into houses, but Farming Simulator could be worth a look. Your kid can probably just explore the map and maybe even try some of the farming.

As someone else mentioned, Slime Rancher might be fun just to explore the world with the cute creatures all around, but there’s not really much or any buildings as far as I know (don’t know if the sequel has this stuff).

A Short Hike could be worth a look. You’re on a mountainous island / national park type deal and your main objective is to get to the peak of the mountain, but you can just explore, do some side quests, solve puzzles, etc. I feel like the screenshots on the Steam Store page for the game look really ugly, since it’s so pixelated, but that’s just a filter, that you can change in game (from smooth to really pixely).

What’s the underwater game that came out a few years ago? Your spaceship crashes in the ocean of a cold planet, and you explore and make bases, and solve a mystery… SubZero or something?

Sounds like “The witcher 3” world will be a good fit for your daughter curiosity, the guest line however is too dark for her age.

If combat is an option, those simple ARPGs could fit the bill nicely:

• Quest Hunter
• Cat Quest 2

No one said Journey, but it’s one of the best immersive games that let you wander in really cool environments. Might be a little frightening in some parts but under your supervision it should be fine