Follow

I spent parts of January working on a new kind of search engine. Instead of indexing the unfathomable internet, it focuses on enabling search for neighbourhoods of related websites—webrings, or other kinds of topically similar sites.

The search engine is called Lieu.
Try it out: lieu.cblgh.org
See the code: github.com/cblgh/lieu

The Lieu instance I host indexes the Merveilles Webring webring.xxiivv.com, a collection of interesting sites with a significant overlap of the folx here in merveilles.town. As such, sites in the ring typically have a shared focus on programming projects, art & note-taking.

Currently, the focus is more on enabling serendipitous discovery than exactly finding what you may be looking for—it is quite simple, at the moment, and benefits smaller queries into your interests. Try tossing in a word and see where you end up :)

Oh yeah, and if you're feeling listless, at least give lieu.cblgh.org/random a couple of clicks—you'll be redirected to one of Lieu's indexed pages, at random.

@cblgh Nice work! This is really cool.

I like how linking /random caused mastodon to scrape from the redirect, so it is showing meta data for xuv.be/writtenImages-follow-up

I wonder if the scrape is the same for everyone? 🤔

Anyway, this is awesome, thanks for putting work into it and sharing :)

@cblgh I will add Lieu to the webring index page footer :D

@cblgh Added a link directly to Lieu's /random page as well. 🔥

@cblgh The search button has these weird little corners, I can't figure out where they're coming from:

@neauoire oh no! i noticed that as well in the favicon, but thought it was my export that added them hmm

@cblgh It's still there, I'll give it a moment to propagate properly.

Could you add a value="search" to the search button on home for accessibility?

@cblgh I wonder if it would be possible to aggregate the pages into a sort of index, showing like..

a table with the top 100 keywords listed in order of the most present. Like, if 3 people have /now pages, then now has a value of 3, if 10 people have /discourse

Discourse________10
Reading________7
Now__________3
...

@neauoire yeah something like that could be interesting, i'll see what makes sense wrt the current data :3

@cblgh id love to learn how to expand the content that it's scraping from my wiki. It looks like it hits the headers but content under those isn't always picked up. Gonna read through the source code soon. Nice work!!✨🍓

@metasyn :tealheart: one safe bet is adding some descriptive text to the <meta name="description"> tag, or some keywords :)

other than that and the headers the first paragraph is important, but that can sometimes be contain stuff like random interface elements / navs due to how some people do markup structure :)

@cblgh hm... any thoughts on if this could have a "search engine API" for the individual sites? Then we could have some minimal JS to get the results for our own site, or jump to Lieu with a site filter?

@ciel yeah that's planned :) i'm thinking that you could just implement the same search form that i have on the main site & add "site:<yourdomain>"–site: & other filters not implemented yet tho

@cblgh it's lovely. the presentation makes it feel like a curated list rather than meaninglessly long google results.

I need to add descriptions to my pages.

@fleeky o ya that's an interesting usecase, i'll have to make add a static build option to monotome then :) i don't think the crawler will do that well on javascript-only pages (such as monotome atm)

@cblgh doh! No rush , but I am quite excited to try out more software from you. Thank you again for all that you do.

@cblgh Lieu is so great *___* I'm so happy you put this together.

@cblgh This is awesome! Though it makes me realize I need to tweak/expand my website's content to display more interesting stuff, as it doesn't seem to look into lists (but maybe I'm wrong.) I definitely need to add actual wiki-like pages anyway.

@ice you're right. unless the list content is somehow represented in the <meta name="description"> / <meta name="keywords"> tags, or a h1-h3/the first paragraph, it won't be crawled. sorry about that!

@cblgh No problem, looks like this is the impulse I needed to make a big update, I’ve been thinking about it for a while :)

@cblgh It turned out beautiful! And so useful, very very cool, thanks!

Sign in to participate in the conversation
Merveilles

Merveilles is a community project aimed at the establishment of new ways of speaking, seeing and organizing information — A culture that seeks augmentation through the arts of engineering and design. A warm welcome to any like-minded people who feel these ideals resonate with them.