pbloem , 2 months ago @futurebird @taylorlorenz Apparently, the index is the real bottleneck. A high quality index is hard to create. I wonder if a decentralized, (SETI@Home style) volunteer effort could work. Everybody runs a little crawler, and the results are unified over DHT. Then we build loads of open source search engines on top of it, filtering however we like.
@futurebird @taylorlorenz
Apparently, the index is the real bottleneck. A high quality index is hard to create.
I wonder if a decentralized, (SETI@Home style) volunteer effort could work. Everybody runs a little crawler, and the results are unified over DHT.
Then we build loads of open source search engines on top of it, filtering however we like.