‘Making Moves’ In DALL·E mini

Learning to make moves with DALL·E mini is about writing prompts that get the best from the subsurface potential in the machine intelligence

10 minutes

The recent explosion in memes and images from DALL·E mini has been awesome to watch. It’s amazing what a clean interface on a highly technical system can do for usability..

I’ve been generating the cover images for Permanently Moved using various iterations of CLIP since last August. But that requires firing up Google Colab, messing with the google drive integration, then prompting it with the podcast description/title etc and waiting an hour or so.

DALL·E mini’s interface in contrast to colab notebooks is clean and super smart.

I love the way it gives you the 3×3 grid of images all self contained on the page. It’s built in sharing functionality also includes a screenshot of the prompt, which is also important.

I was wondering this week, what I find so entertaining about the DALL·E mini image shares online. Then I realised that they remind me of political compass memes. Not just visually – I think that much is obvious:

DALL·E mini Prompt: ‘Multispecies’

But also the way they work, cognitively: When I wrote about political compass memes a few years ago I quoted Ahmet A. Sabancı (who you’ll remember I saw IRL last week).

Here’s two quotes from Ahmet about compass memes, that also apply to the visual representation of DALL·E mini in the feed:

one powerful side of compasses is the fact that it gives all options/possibilities at once, overloading the mind and forcing you to think about it.
Compasses —especially the ones like the one here— with their design and the information density is something perfect for our current social media madness. It interrupts the clickbait and will-be-forgotten-tomorrow news and never-ending outrages with its weirdness and pulls people out of it.

If (like me) you think of ideas, narratives and memes as containers, then the DALL·E mini images shared online are really information dense containers.

They show 9 images, or 9 rather possible answers that the AI has given to the prompt. In the same way as Compass Memes for a time disrupt the day to day clickbait of the feed, so too do DALL·E mini’s images.

What does a pigeon giving a TED talk look like?

Well… here’s a visually dense answer to just that question.

The key element (I think) to the enjoyment, and understanding of these images is the inclusion of the prompt. It contextualises the weird dreamscape images. Or rather the world the prompt suggests.

Last year on Permanently Moved I talked about the Prompt in Interactive Fiction and AI Art and the similarities between the two.

To work with a neural net and its outputs is like spelunking or steering a boat under its own power. Like walking through dark hallways or moving through symbolic space.

You put in a prompt and see what unfolds.

This of course is similar to engaging with interactive fiction. In both cases one is embedded within a system that has a model of the world. 

I’ve been writing and thinking a lot about ‘Expression’ in worlds this week.

In the example of a compass meme, in order to understand whats ‘expressed’ you need to look at the axis, and read all the individual squares to understand the ‘whole’ . To understand whats expressed by the 9 DALL·E mini images, you need to look at each one AND read the prompt to contextualise the whole.

The prompt is the most important element of interacting with algorithmic systems. The ability to explore the ‘world’ created by the algorithm returns agency to ‘Interactor’ or user. A completely diffrent experience to being trapped inside an algorithmic new feed in the ‘world’ of Twitter.

This is summed up nicely in Mary Ann Buckles’ 1985 dissertation ‘Interactive Fiction: The Computer Storygame “Adventure.” 

One of the fascinating aspects of Adventure is that the reader must figure out how the game and narrative works, i.e.

1. How to communicate with the computer/narrator
2. How to make moves

Learning to ‘make moves’ with DALL·E mini is about learning how to write prompts that get the best from the subsurface potential of the machine intelligence you are interacting with. The prompt both represents ‘the surface of the world’, and the thing that abracadabra’s it into existence.

The summoning words that set the world in motion. 

The command line or parser is so tightly bound with the Interactor and the game world that they are indistinguishable.

We explore and ‘make moves’ in the algorithmic space of DALL·E mini’s world model by prompting it.

Sometimes we can make moves in ‘DALL·E space’ that will consistently return ideas that don’t exist in real space:

Check out the whole Crungus thread. Whats going on here is that a word/prompt is exposing the shape/or topolgy of DALL·E mini’s possibility space. In Buckles language (speaking about interactive fiction) it’s exploring how both the ‘game and narrative works’ in real time.

See also Janelle Shane writing last year on AI Weirdness.com: ‘My favorite nonexistent painter’

But I also happened to experiment with made-up artists, and surprisingly they had distinct styles.

Here’s “Internet Infrastructure by Carmine Nottyors”.

She has never existed. She paints a mean tangle of wires.

I’m intrigued that CLIP+VQGAN can generate unique painting styles for nonexistent artists. It’s a dimension of control that I don’t think I’ve seen people exploring yet.

Lastly, whilst I’m on this train of thought to nowhere.

A couple of weeks ago, Mat and Holly posted a long essay called ‘Infinite Images and the latent camera’.

They wrote:

DALL·E represents a shift from attempts to reflect objective reality to subjective play. Language is the lens by which we reveal the objective reality known to the neural network being explored. It is a latent(hidden) camera, uncovering snapshots of a vast and complex latent space.

This exploration is what I meant by ‘making moves’ last year. It is the exploration of, but also the creation of worlds.

As these systems become more advanced, the possibilities for the co-creation of worlds becomes more and more exciting.

Once the friction to share what is on your mind has been eliminated, the ability to co-create social narrative art experiences at the dinner table or the theatre seems conceivable and exciting!

I can imagine ‘world models’ as ‘Permissive IPs‘.

It seems like a neat way of solving some of the IP issues. Prompt the world model to expand your little corner of the world you’re ‘playing’ in. Power Fandoms can then collectively get inside the world model and fully explore the ideas contained inside the latent possibility space of a custom DALL·E mini system.

Handing a GAN/world model over to an audience/fandom/DAO/writers room is a sort of wind-up world with infinite permutations of expression.

This sort of co-created exploration is the sort of world that would be navigated by lore.


Speaking of clean interfaces on highly technical systems…. and the reason I started writing this post in the first place. Here’s a LIZARD! meme that came to me in a dream that I posted to Tumblr earlier this week.

When DALL·E mini style GANs are integrated straight into meme templates its gonna be a new golden age for memes!

Photo 365

165/365

Permanently Moved

Win Condition

The ultimate victory condition in the most popular online worlds right now is to be left alone.

The Ministry Of My Own Labour

  • Reading
  • Writing
  • Arithmetic
  • lol
  • Calls with Ross about New Centre Course
    • Prepping for a recorded/printed dialogue with Mat and Reza.

Dipping the Stacks

The big idea: have we been getting sleep all wrong? | Health, mind and body books | The Guardian

Eight hours a day is a myth. Embracing our individual sleeping patterns could be the key to a better night’s rest

AI Inventing Its Own Culture, Passing It On to Humans, Sociologists Find

Algorithms could increasingly influence human culture, even though we don’t have a good understanding of how they interact with us or each other.

could and increasingly” SMDH

Every 30 hours, world gets a new billionaire, a million new poor

The question no longer is why poor remain poor, but why rich get richer this fast

We Are Not Living in a Simulation, We Are Living In the Past | The Convivial Society

On the internet, there is no present, only variously organized fragments of the past.

Temporal Belonging — Real Life

The timeless, futile effort to fix circadian rhythms with tech

Reading

I finished listening to the Audio Novella Bookkeepers Skull by Justin D Hill. It’s a real ‘mood horror’ piece. Sci-fi farmstead, warhammer grimdark, mysterious murders. Really quite good.

I’m reading Orthodox Worship: A Living Continuity with the Synagogue, the Temple, and the Early Church by Benjamin D. Williams. This is a book in two halves. The first 100 pages covers the history of christian worship from its Jewish and pre-Christian influences, then explains/covers the development into the Orthodox Liturgy – focusing on how early it developed and stressing that the Liturgy has not changed. The second half of the book analyses the ‘shape’ of the Orthodox Divine Liturgy, from start to finish. I’m nearly done.

I read all the other authors contributions to Lost Zone. Hiking the Dawn of Metaverse. I think its a fantastic book (and not just because I’m in it). If you are interested in virtual worlds and their design in 2022 consider checking it out!

I’d been saving The Last Ditch audiobook book 7 in the Ciaphas Cain series in the Warhammer 40k universe. I only started listening whilst making breakfast this morning, but as always its highly entertaining. I also feel like the ‘ridiculous’ dials have been turned up?

Music

thejaymo.net Spotify Playlist

Dariacore 3​.​.​. At least I think that’s what it’s called? – c0ncern

Well, the third and final instalment of leroy/dltzk/c0ncern’s/Zeke trilogy of ‘Dariacore’ albums is out. What a fitting end to a totally made up genre. There is now coming back from this frontier however, pop music has been forever changed. Culture has been infected, 100 Gecs seem down right pedestrian in comparison.

For those unfamiliar Dariacore was invented (and exploded) during the pandemic. On paper it’s one of the most extreme genres built on samples of other songs. Think 100 gecs but chopped up existing pop songs turned up to 11. In practice it is some of the most aggressive digi-core/break-pop ever committed to mp3.

The opening track below on this final album gives you a good idea of what you’re in for. Not just the vibe but for the whole album. Absolutely punishing popbreaks, hyperactive sonic jumpcuts, super sped up vocals, 5 musical ideas per minute.

It’s what happens when mashups get so far out on a limb that they bend, hit the soil and become their own tree. A tree that pulverises raw sonic material into new green shoots.

I can’t express how much I like this album.

I think Dariacore 3, and to a lesser extent the hyperpop-adjacent digi-core sounds like the experience of being on the Internet in 2022. It sounds like the future.

I spent years in the late 2000’s playing obnoxious noise and breakcore at people on stage at parties and venues. I truly belive I was doing a sort of missionary work – bringing the sound of the future to people. Not that I knew it at the time, but the 200+ bpm anarchist breakcore I was making and sharing online was in part laying the ground work for genre transcending work like this.

Dariacore and breakpop in general floats my boat. I’d (and do) listen to this shit for fucking hours, all day every day. Its not for everyone I’ll admit, but it is the sound of the future, wether you like it or not.

We need to live in a world where this shit gets on regular rotation on the radio.

Melt some boomer brains.

Remember Kids:

Prefer Email? 📨

Subscribe to receive new posts straight to your inbox!

Or subscribe to my physical mailing list from £5 a month


Leave a Comment 💬

Click to Expand

5 responses to “‘Making Moves’ In DALL·E mini”

  1. […] 21-29) I talked about my first tentative steps with AI models and image generators. I’ve had thoughts over the last 12 months. Recently I’ve come to the conclusion that the short term impact of these […]

  2. […] bahsettiğimiz teknolojiler için kilit noktanın bizim hayal edip verdiğimiz komutlar olduğunu unutmamak lazım. Bizim hayal gücümüz ve dili kullanma biçimimiz bu araçların üretebileceği şeylerin […]

  3. […] to the character and image i’m trying to replicate it looks like shit. But we’re ‘making moves’ in latent space here. Need to learn to dance before you can do the […]

  4. […] as only predicting the next word. But what the next word is depends on where you are in the model. Make a move from one place to another in latent space and you’ll get a different set of next […]

  5. […] weights and attention mechanisms are all constraining and domesticating forces on the wild robot, or latent space that lies beneath the surface. Prompt injection is the bribing and tricking of the wild robot […]

Leave a Reply

Your email address will not be published. Required fields are marked *