We need decentralized, federated search. I remember YaCy from years ago was attempting this. Anybody know if there’s anybody actively working on this?
@makeasnek @schizoidman YaCy is still around.
And https://searx.space/ is an open source metasearch search engine with many instances. (Try https://searx.be/ if you want to test it out.)
SearX/SearXNG allows you to aggregate results from a number of different search engines. You choose which ones, and they’re stored in your browser without setting up an account.
Interesting thanks I’ll take a look!
@makeasnek On a broader note, I think possibly the best approach for decentralised, open-sourced web search might be an evolution on the SearXNG model.
At the top of the funnel, you have meta search engines that query and aggregate results from a number of smaller niche search engines.
The metasearch engines are open source, anyone with a spare server or a web hosting account can spin one up.
For some larger sites that are trustworthy, such as Wikipedia, the site’s own search engine might be what’s queried.
For the Fediverse and other similar federated networks, the query is fed through a trusted node on the network.
And then there’s a host of smaller niche search engines, which only crawl and index pages on a small number of websites vetted and curated by a human.
(Perhaps on a particular topic? Or a local library or university might curate a list of notable local websites?)
(Alternatively, it might be that a crawler for a web index like Curlie.org only crawls websites chosen by its topic moderators.)
In this manner, you could build a decent web search engine without needing the scale of Google or Microsoft.
@ajsadauskas sounds like what @pears is trying to do
@makeasnek @pears is working on that
The AI bullcrap at the top, before actual search results, is what finally made me change my default engine to duckduckgo
Same, I have tried DDG every once in a while but kept going back to google. Now google search doesn’t quite give me relevant results anymore and all the AI crap just takes all of their effort to work on search itself.
Been using DDG for a few weeks now for personal and work related stuff - quite happy with it.
Reid characterized the mistakes that won attention as the result of an internet-wide audit that wasn’t always well intended.
Oh yeah, there was a conspiracy to embarrass Google with off-the-wall questions like “how do I thicken pizza sauce?”
Why are these corporate schmucks unable to take any blame?
No audit is well intended. The whole ppint of an audit is to find duplicitous and fucked up shit.
making the feature rely less heavily on user-generated content from sites like Reddit
Imagine selling out reddit/buying access to the comments for AI just to immediately unprioritise it
A new deal is being forged with 4chan instead.
No one could have seen this coming. Obviously LLMs absolutely completely understand the difference between people joking around with each other and authentic advice.
Sarcasm aside, Reddit does have some good information about niche topics. There’s just currently no way for AI to understand the difference between dry humor and serious responses. I think the AI summaries are unhelpful anyway, but even if I didn’t it’s pretty obvious Google didn’t think further than “Shove AI into it like a drunken prom night encounter”.
The company made “more than a dozen technical improvements” to AI Overviews …
… making the feature rely less heavily on user-generated content from sites like Reddit
So it prefers the results that Google normally deprioritizes? I guess we have that in common
I almost wonder what Huffman is thinking right now - he could have been so very close to becoming a billionaire, managing the repository of human general technical knowledge explained simply, but he blew it by being a greedy piggy…
Not that he will ever admit that, even to himself.
I don’t think Spaz is thinking at all. That was how Reddit imploded.
His decision tree is like: I say yes, content creators say no, admin rights say yes tho, market ends up saying no tho, the end.
It seems like I’ve heard this story before somewhere lately…
It’s an interesting concept. But right now, Google Search is the best argument in favor of Bing.
Or DuckDuckGo, which uses the same index as Bing, but with more privacy
You think Bing aka Microsoft is not planning on this exact same folly?
I’m sure they are. But they haven’t yet. And after this, they might at least borrow a page from Apple and make sure it works first.
Like Apple Maps? Or mms messaging?
Nobody’s perfect.
I don’t know of any MMS problem, but I also don’t generally text outside of iMessages. Maps was a cockup, and Apple owned it.
Ok good. Now remove that feature or at least add mandatory human review
Human review is NEVER going to happen. The amount of comments from Reddit and user queries is mind boggling
No I meant that the AI makes replies to most popular (or predicted to be popular) questions but a human has to review and accept it manually to push it to the search results. But that still won’t fix everything. It’s just AI that needs to die. That’s it
And slow down search even more?
