Conversations with an AI - part 4

In part 1 of this exploration I introduced a fantastical story by the Polish writer Stanislaw Lem. It describes how an inventor called Trurl constructs an electronic bard — a poetry-writing machine — and invites his arch-rival, Klapaucius to set it a challenge. At first the machine falters, producing little more than incomprehensible gobbledygook but when later, following a few adjustments, it produces a witty and poetic put-down, Klapaucius becomes enraged and decides to set it an impossible challenge: 

Alright, he says:

"Have it compose a poem — a poem about a haircut! But lofty, noble, tragic, timeless, full of love, treachery, retribution, quiet heroism in the face of certain doom! Six lines, cleverly rhymed and every word beginning with the letter s!"

To which Trurl protests: 

"You can’t give it such idiotic …"

But he didn’t finish. A melodious voice filled the hall with the following: 

Seduced, shaggy Samson snored.

She scissored short. Sorely shorn,

Soon shackled slave, Samson sighed,

Silently scheming, 

Sightlessly seeking

Some savage, spectacular suicide.

I love this story because, of course, in reality, the remarkable poem was clearly dreamed up in advance of the impossible specification — Lem being only human, after all. 

As the story has it however, the machine responds to Klapaucius’s outrageous request with a level of intelligence and creativity that far exceeds the capability of a human poet. It is as if we find ourselves presented with a benchmark defining the level a future machine would need to attain if it were ever to be considered truly intelligent. 

And yet, whichever way round you prefer it: specification before poem, poem before specification, the fact is, this is unmistakably the product of a human imagination. As things stand it is literally inconceivable that the tale of Trurl’s Electronic Bard could have been written by a machine. 

All the same, I decided it would be interesting to set GPT-3 a simple version of the same challenge. I would prompt it with the haircut poem and two other tautograms (this being the term for texts in which every word begins with the same letter).

Here is what I typed in: 

1. Poem about a haircut in which every word begins with s

(see above)

 

2. Poem about love in which every word begins with h

His heart humbly hailed happily. 

Her hatred haunted his heart. 

He hated himself. 

Helpless, homeless, hopeless!

He held her hands, 

Healed her heart.

Her happiness healed his hurt

3. Poem about apples in which every word begins with a

All alone, an afternoon ago, 

Adam ate apples, already aplenty

Aided astounding appetite, 

Aiming at awful aches alive

The last two poems are the best I could come up with after a brief online search. They are nowhere near as good as the first one. Would GPT-3 do any better, I wondered. I didn’t suggest a topic but left it to suggest one of its own. Here are two of the better outcomes which, it has to be said, needed sifting out from a quantity of lower quality dross.  

4. Poem about my father in which every word begins with b

Big-bearded bard in beret 

Busily bashing bricks 

Bade me balance books 

Battled bards and bibles

Bold bully, brown baker, 

Beseeching beneficiaries

and …

4. Poem about a wedding in which every word begins with r 

Religious ritual 
Reverently rearranged 
Received the ring 
Ran a red light 
Ran away with the bride 
Relaxed, radiant, 
Received the rice 

Not bad, I thought. 

Later I tried suggesting topics. So prompted with:

4. Poem about death in which every word begins with e

GPT-3 produced: 

End came early, unnoticed 

Enduring until envied 

Ending in utter emptiness.

You might have noticed, in the last two examples, that it is not too fussy about sticking to the single letter stipulation. It’s as if it doesn’t get just how important this is. 

You may have noticed that I am beginning to talk about GPT-3 in a way that implies it has a will of its own. I don't actually believe it does of course; it is more a case of not having the language or framework to discuss its behaviour in any other way. 

Occasionally it tosses out a few extra lines which do not obviously connect with what has gone before. Like this:

A boy named Bobby bought a ball. 

The ball was blue and bounced.  

This one continues to bounce around in my memory - quite possibly on account of the fact that the couplet would be vastly improved by replacing bounced with bouncy. 

But then, as they say - nobody’s perfect. 

Published by David Wilson under OpenAI API Community Guidelines 

Conversations with an AI - part 3

So I have been trying to persuade GPT-3 (an artificial intelligence) to write poetry.

Sorry about the wait, by the way, but it comes with the territory. Omnivorism being essentially all about the pursuit of a multiplicity of interests, there is  always the risk I will lose myself down ‘The Garden of Forking Paths’ or even down the real garden, picking apples, building vegetable frames or otherwise savouring the last honeyed days of autumn.

But back to the matter in hand.

If I understand it correctly, the best way to persuade GPT-3 to do something is to give it an example of what you are after. So, in order to get it to write poetry, I tried the following prompt:

1. Poem about love 

I ne’er was struck before that hour

With love so sudden and so sweet, 

Her face it bloomed like a sweet flower 

And stole my heart away complete

2. Poem about daffodils 

I wandered lonely as a cloud 

That floats on high o'er vales and hills, 

When all at once I saw a crowd, 

A host of golden daffodils; 

Beside the lake, beneath the trees, 

Fluttering and dancing in the breeze.

 

3. Poem about Autumn 

Season of mists and mellow fruitfulness, 

Close bosom-friend of the maturing sun; 

Conspiring with him how to load and bless 

With fruit the vines that round the thatch-eves run; 

To bend with apples the moss’d cottage-trees, 

And fill all fruit with ripeness to the core 

4. Poem about a beautiful sunset. 

The numbers help identify a repeating pattern made up of a single line indicating that this is a poem (whatever that is) about something, followed by the poem itself. 

GPT-3 completed the pattern with:

Deep-thundering drove of serried clouds, 

Purple-black like distance of violets, 

Like purple-black grapes 

On pallid old vines, 

Under a plum-black sky, yonder 

Pile huge, sharp shadows on the hills.

What is impressive here is that GPT-3 appears to have recognised both the structure (a poem with the right sort of shape and rhythm) and the asked-for content (a sunset) despite the fact that the actual word sunset does not actually appear in the response

I tried Googling some of these lines and came up with a blank, so I am assuming they are ‘original’. However after changing the last part of the prompt to: 

4. Poem about the moon

I got the following:  

White in the moon the long road lies, 

The moon stands blank above; 

White in the moon the long road lies 

That leads me from my love.  

Which is a poem by A. E. Housman. 

Obviously GPT-3 considered this a perfectly reasonable response, and who can blame it? Left to itself, it appears entirely relaxed when it comes to plagiarism. If there is a way to discourage this, I have no idea what it is. 

I tried the same prompt again, with the following result:

O, never say that I was false of heart, 

Though absence seem'd my flame to qualify. 

Perfectness I pursued that I might merit thy loves: 

but, O, what menace, fears, what anguish, 

Mine own! my eaglets, now at distance soar!

Apart from the fact that this would appear to have very little to do with the moon, a quick Google reveals the first two lines to be a straight copy of Shakespeare’s sonnet 109. But quite where those soaring eaglets come in is anyone’s guess. 

Finally, asking GPT-3 to write a poem about rain, it responded with: 

Rain, gentle rain, 

Thy tiny drops create 

A world with rivers, roads and shores, 

To shine and charm our sea of woes.

Which is just good enough to be impressive and bad enough to be genuine.

I am nearing the end of this exploration but it wouldn't be complete without a return to Lem's story of The Electronic Bard  and that is what I propose to do in the next part. 

Published by David Wilson under OpenAI API Community Guidelines 

Conversations with an AI - part 2

I have been interacting with an advanced natural language system called GPT-3 and decided I would invite it to write poetry. You can read Part 1 here. 

As a first experiment, I logged into the GPT-3 API and typed: 

Write a poem about a sunset 

To which it responded:

A poem? Really?  

I’m supposed to write a poem about a sunset?   

What kind of sunset?  

What does it mean?  

Why do I care?  

I guess the problem is that I don’t know how to write a poem.  

My mind is a blank. 

I’m stuck trying to figure out what I’m supposed to do. 

I’m supposed to write a poem about a sunset? Seriously?  

I guess it’s not a big deal. 

Maybe I’ll do it. But, maybe I won’t.  

Maybe instead, I’ll write a poem about how to write a poem.

What is going on here ? On the face of it, it would appear GPT-3 has ‘understood’ my prompt as a command to write a poem — the key word being write, taken as an imperative, write! And the response from GPT-3 implies it is vaguely insulted — this, after all, being our first interchange. Maybe I should have introduced myself first; maybe I should have said ‘please’. 

All of which is totally misleading as an explanation of what is going on. 

Rather than thinking of the prompt 'Write a poem about a sunset' as a command, instruction or invitation it is more helpful to think of it simply as a collection of words, or fragments of words and, when I press <return> and invite GPT-3 to respond, I am, in effect, asking it:  

'What does this remind you of ?'

So in the case, 'Write a poem about a sunset' the system, drawing on its gargantuan store of data, is reminded of the kind of response that might be associated with the sort of curt, impolite command I gave it. That, at least, is the kind of intuitive mental model that I find useful; I can’t claim to understand how a text-based autoregressive language model actually works. 

In any case, I thought I would try the polite approach and typed: 

Please would you write a poem about a sunset

And, this time, got the response: 

I don’t know if there are any rules about what has to be in a poem   

But I think this might be a good one:  

Sunset 

The sun has gone to sleep. 

The orange sky is fading.

Wow, that’s better. But who knows whether that ‘please’ made any difference, because GPT-3 rarely repeats itself. Presented with exactly the same request a second time, it responds with: 

So that’s all it takes to get poets to sit down and write poems? A polite request?

Which is also pretty impressive and if my intention was to dazzle you with GPT-3’s abilities I might be tempted to leave it there. But that was only the first line of its response; it went on to say:

And then he’s jobbed and all those poems about sunsets are being read at his 

memorial service. And he, this fallen and deposed man, has become once again, 

I guess, the real poet. 

The problem with Meyer’s TED-speech idea is that it trivializes the poet as a 

mere vehicle for his words. “But what if you gave a lecture and nobody came?”
It’s not the poet who’d suffer, but PR. And it’s PR who we’d be trivializing, 

because it’s PR who’d be making the TED-talk poet into a figure of fun.
And with this revolution for creative writers, the poet would be liberated from 

the banality of the poet’s life. 

This is vaguely intelligible but, as a response to the original request, it makes no sense. Instead it appears to be a mash-up of a discussion of a TED-talk in which poems, sunset and politeness feature. Furthermore, it is highly likely that some time spent googling ‘Meyer TED-talk’ or suchlike would reveal where all of this came from. 

But it’s not the lack of originality that is the problem here but the failure to address the intention expressed in the original prompt. Given that I am trying to persuade GPT-3 to write poetry, we're not doing too well. 

Fortunately, there's a better way - as I will explain next time. 

To be continued … 

Published by David Wilson under OpenAI API Community Guidelines

Conversations with an AI - part 1

You may have heard about GPT-3, an artificial intelligence (AI) system developed by the San Francisco based company, OpenAI.

 

Technically speaking, GPT-3 is a text-based autoregressive language model that has been trained by being exposed to an enormous volume of web-based data. To interact with it you supply a textual prompt and it responds with a textual completion. The things that set GPT-3 apart are both its vast scope with respect to subject matter and the remarkable quality of its responses - which can be easily mistaken for texts written by humans. This has led a number of people — including members of the GPT-3 research team — to acknowledge the potential for misinformation, fraudulent academic essay writing and other forms of misuse and to investigate ways to mitigate these risks.

 

In June 2020 OpenAI invited users to request access to its user-friendly API in order to help them "explore the strengths and limits" of this new technology. I was delighted to be granted beta access to the API two weeks ago. However, the question immediately arose: what was I going to ask it to do? 

                                                                  ☙

One of my favourite books is the Cyberiad by the Polish author Stanislaw Lem — a collection of short comic pieces describing the antics of rival constructors, Trurl and Klapaucius.

If you were to look for a copy of the Cyberiad in a secondhand bookshop you’d find it in the science fiction section, despite the fact that the book resists categorisation. It was written in 1965.

In one story — Trurl’s Electronic Bard — the constructor sets about building a machine capable of writing poetry. It soon becomes clear however that, to be successful, the machine will first need to be programmed with the sum total of human civilisation and culture, since great poetry can’t be expected to emerge from anything less.

So, in the story, the constructor works away for a number of years and, after several frustrating setbacks, he invites his arch rival Klapaucius round for a demonstration. At first the results are disappointing and Klapaucius delights in Trurl’s discomfiture, rolling on the floor laughing as he watches him frantically adjusting the settings. But then suddenly, as Trurl rushes back and forth, there is a crackle, a clack and the machine with perfect poise says:

 

The petty and the small 

Are overcome with gall 

When Genius, having faltered, fails to fall.

 

So that decided it. I would encourage GPT-3 to write poetry.

To be continued … 

Published by David Wilson under OpenAI API Community Guidelines