With the frustrations from using the usual search engines growing I’ve been thinking about this a lot lately.
It seems we’ve poisoned the well by allowing the proliferation of advertising interests to dominate the web.
Like how hard would it be to make your own non-commercial index?
Only human made sites that aren’t related to buying, selling, marketing, etc.
Could that be a federated open-source project?
With the open source tools that are available nowadays I think it’s not necessary hard, but you shouldn’t underestimate the hardware costs required to do this well. You will need a lot of high speed disk space to store the index and you will need an army of crawlers to stay up to date.
Interesting. I wonder if we’ve finally found a use for blockchain.
*Edit: lol, the downvotes. Sheesh
We haven’t
There’s a reason why only the big few are doing it. Even DuckDuckGo have given up their own crawler long ago.
What’s the reason?
spin up YaCY, see for yourself. It turns into a full time project, because there’s always ranking tweaks and and indexing startpoint changes to be made.
It’s relatively easy to put together a crawler for specific purposes, but a general web search is hard, and requires a lot of storage.
I might give it a whirl. Thanks!