I spent parts of January working on a new kind of search engine. Instead of indexing the unfathomable internet, it focuses on enabling search for neighbourhoods of related websites—webrings, or other kinds of topically similar sites.
Oh yeah, and if you're feeling listless, at least give https://lieu.cblgh.org/random a couple of clicks—you'll be redirected to one of Lieu's indexed pages, at random.
@cblgh Nice work! This is really cool.
I like how linking /random caused mastodon to scrape from the redirect, so it is showing meta data for https://xuv.be/writtenImages-follow-up.html
I wonder if the scrape is the same for everyone? 🤔
Anyway, this is awesome, thanks for putting work into it and sharing :)
@cblgh The search button has these weird little corners, I can't figure out where they're coming from:
@neauoire oh no! i noticed that as well in the favicon, but thought it was my export that added them hmm
@cblgh It's still there, I'll give it a moment to propagate properly.
Could you add a value="search" to the search button on home for accessibility?
@cblgh I wonder if it would be possible to aggregate the pages into a sort of index, showing like..
a table with the top 100 keywords listed in order of the most present. Like, if 3 people have /now pages, then now has a value of 3, if 10 people have /discourse
@neauoire yeah something like that could be interesting, i'll see what makes sense wrt the current data :3
@cblgh id love to learn how to expand the content that it's scraping from my wiki. It looks like it hits the headers but content under those isn't always picked up. Gonna read through the source code soon. Nice work!!✨🍓
@metasyn one safe bet is adding some descriptive text to the <meta name="description"> tag, or some keywords :)
other than that and the headers the first paragraph is important, but that can sometimes be contain stuff like random interface elements / navs due to how some people do markup structure :)
@cblgh hm... any thoughts on if this could have a "search engine API" for the individual sites? Then we could have some minimal JS to get the results for our own site, or jump to Lieu with a site filter?
@ciel yeah that's planned :) i'm thinking that you could just implement the same search form that i have on the main site & add "site:<yourdomain>"–site: & other filters not implemented yet tho
@cblgh it's lovely. the presentation makes it feel like a curated list rather than meaninglessly long google results.
I need to add descriptions to my pages.
@cblgh doh! No rush , but I am quite excited to try out more software from you. Thank you again for all that you do.
@cblgh This is awesome! Though it makes me realize I need to tweak/expand my website's content to display more interesting stuff, as it doesn't seem to look into lists (but maybe I'm wrong.) I definitely need to add actual wiki-like pages anyway.
@ice you're right. unless the list content is somehow represented in the <meta name="description"> / <meta name="keywords"> tags, or a h1-h3/the first paragraph, it won't be crawled. sorry about that!
@cblgh No problem, looks like this is the impulse I needed to make a big update, I’ve been thinking about it for a while :)
Merveilles is a community project aimed at the establishment of new ways of speaking, seeing and organizing information — A culture that seeks augmentation through the arts of engineering and design. A warm welcome to any like-minded people who feel these ideals resonate with them.