Experimenting with CLIP+VQGAN to create visual mnemonics for chinese characters


Some more examples, trying to give the neural network a bit more of a challenge with some of these

"spirit", "quick", "wood"
"I / me", "rabbit", "mouse"

I think my favourite here has to be "兔" with the little bunny sitting on top!

· · Web · 4 · 8 · 23

I have to say, one of the funniest results I got was one of the variations of 快 (quick)

As the text prompt I gave the definition of the word "quick", and this ended up transforming the character into a person running with the word "FAST" written the background

Unfortunately despite mostly matching up with the shape of 快, it is not very readable as a chinese character on its own without context

Some more examples of these pictographic pictograms with characters for different animals

I think I'm starting to get more reliably good results

These are the last three characters I generated-- showed them to a few friends who do not know any chinese and they all guessed the meanings (or in the case of 电,the prompt that I gave it) with 100% accuracy

It makes me think that I could make a tool for learning chinese where first it presents this visual styled version of the character before telling the learner the meaning

I think this will make learning more active, first you guess what the meaning of each character is, and then when the defition is revealed, likely confirming what you've already come up with, which will make the character even more memorable because you'll have been an active participant in the process

@neauoire Ask and ye shall receive:

Green, blue, purple, red, orange, yellow, white, gray, and black

Interesting how each leans into its own associations in different ways: orange with autumn leaves, green with plants (and aurora borealis in the top-left?), gray with... layered rock formations?

@neauoire correction: top-right*

I think the green one might be one of the most memorable ones here with the leaves, tree, and aurora borealis

This image will certainly be helpful for me to remember because I often mix up the characters for blue/green myself!

@neauoire I also let yellow (黄) go for a few hundred more iterations and came back to find that it had transformed into what looks like some strange, alien user-interface

@azlen I'm fascinated by how 我 looks - It looks like a self portrait by Picasso. I see a hand and half a face in there.

@changbai Interesting! To be honest I can't see any face myself in that one but I found it fascinating nonetheless

@azlen pareidolia strikes again. then "I" am the reason why I see faces in “我”...

@azlen these wre beautiful even if some would not make a great mnemonic.

@azlen i just remembered seeing something like this, and wondering just under my conscious awareness whether you had been inspired by this poster series from pixar for their ibside out film

@zens Oh neat, I had not seen this!

Originally the inspiration actually did come from thinking about mnemonics-- really I started learning Chinese a while back as a testbed for experimenting with learning techniques

However the idea resurfaced recently when I came across this Motion Type Project by Ting-An Ho-- so I'd certainly credit this as part of the inspiration


Sign in to participate in the conversation

Revel in the marvels of the universe. We are a collective of forward-thinking individuals who strive to better ourselves and our surroundings through constant creation. We express ourselves through music, art, games, and writing. We also put great value in play. A warm welcome to any like-minded people who feel these ideals resonate with them.