Assessing the impact of generative AI and NLP on art and literature.
Introduction
As AI technology continues to develop, it is increasingly being into question the bounders between human uniqueness and machines. This subcategory of AI, generative AI, "refers to unsupervised and semi-supervised machine learning algorithms that enable computers to use existing content like text, audio and video files, images, and even code to create new possible content." (AltexSoft, 2022). These algorithms are changing fields of art and literature.
The discourse around this technology is highly polarised with those in favour stating that “this AI has no idea what’s culturally relevant or what is politically relevant or whatever it is that is currently important in the zeitgeist. It’s a mindless but very intelligent music creation system.” (Thompson, 2019) On the other hand, those against such developments fear that it will strip the role of humans and what it entails to be uniquely human.
In this project, the impact of generative AI particularly through the use of NLP on art and literature will be examined. It will also look at what the concerns are around this topic and the validity of these concerns.
Art
In the remit of art, AI is transforming the landscape. Software such as DALL-E 2 and Stable Diffusion are able to transform text into images. These software can create an image simply with a few lines of code. In the case of Stable Diffusion, this code is as follows:
# note for Kaggle, the accelerator has been set to GPU T4x2. torch (Cuba) function doesn't run otherwise
!pip install --upgrade git+https://github.com/huggingface/diffusers.git transformers accelerate scipy
# import libraries
from diffusers import StableDiffusionPipeline, EulerDiscreteScheduler
import torch
from torch import autocast
#set up pipe to connect and run stable diffusion
model_id = "stabilityai/stable-diffusion-2"scheduler = EulerDiscreteScheduler.from_pretrained(model_id, subfolder="scheduler")
pipe = StableDiffusionPipeline.from_pretrained(model_id, scheduler=scheduler, revision="fp16", torch_dtype=torch.float16)
pipe = pipe.to("cuda")
Setting the prompt and image size to the following creates image:
prompt = 'A steampunk woman reading a book with her cat at the bar'
image = pipe(prompt, height=768, width=768).images[0]
imageArt may be in the eye of the beholder, however, one may regard some of these images as requiring some further hours of practice to be regarded as an masterpiece.
Dall-E 2, which became upon for the public without a waiting list in September 2022 (Darby, 2022), produces somewhat better images and includes the following:
Both DALL-E 2 and Stable Diffusion uses NLP to tokenise a prompt, vectorizes it and through the use of recurrent neural networks generates an image, comparing the resulting vector to a dataset of pre-existing images and labels (Tilbe, 2022). This dataset and its quality has been brought into question as artists have found that their copyrighted images gave been utilised. Moreover, Clarke (2022) reports in her article:
"It’s not just artworks: analysis of the training database for Stable Diffusion has revealed it also sucked up private medical photography, photos of members of the public (sometimes alongside their full names), and pornography."
Solutions to such issues have yet to be established and remains a point of concern. That said, in the use of both DALL-E 2 and Stable Diffusion, humans remain integral as key words need to be feed to the AI for it to create art. Subsequently, as highlighted in the video by Vox, new artists have emerged who use AI to create their art and their skill lies in their ability to create the most effective prompts that push the AI to its creative limits.
Literature & Writing
In terms of literature and written text generative AI has been used to create new novels as well as other documents. One major example of such texts is the novel, Harry Potter and the Portrait of What Looked Like a Large Pile of Ash.
Created by Botnik, a machine entertained company, uses computers to remix text. It used NLP and generative AI to create a new Harry Potter novel drawing upon the 7 previously published books which have been fed into the model Voicebox, "which determines the high-frequency words and incorporates them into a 'predictive writer' that resembles the text message screen on a phone." (Tobias, 2018). This process produces funny text such as:
“He saw Harry and immediately began to eat Hermione’s family. Ron’s Ron shirt was just as bad as Ron himself. ‘If you two can’t clump happily, I’m going to get aggressive,’ confessed the reasonable Hermione.”
It should be emphasised that the purpose of this text and of Botnik who created it is in essence create a new form of art, rather than to replace it.
“We would like, selfishly, not to replace humanity with algorithms. instead, we want to find natural ways for people and machines to interact to create what neither would have created alone.” (Flood, 2017)
More serious attempts have also been made to use AI in the creation of stories and includes the attempt by Marche (2017). This system requires a large quantity of pre-existing text and even then received negative reviews from experienced editors.
Creation of Generative AIs
As indicated previously, the process to make such AIs is to use pre-existing piece of work and train an AI to assess how the author uses a set of words. It is then asked to create text with new information. A number of NLP libraries exist and they typically utilise deep learning recurrent neural network or RNN to conduct such predictions. These "are a class of neural networks that allow previous outputs to be used as inputs while having hidden states." (Amidi and Amidi, 2021)
For example, LSTM (Long short term memory units) which uses a cell state and forget gate which eliminates some information which is considered not important and stores some key information which is to be passed to the next step. Its capabilities can be viewed by applying it to an existing text.
The novel Sanditon is an unfinished novel by Jane Austin. This now copyright free text was written in 1817 and consists of 24,000 words. (Project Gutenberg Australia, 2013) The tale, in its early stages, lacks a robust plot though the world has been established.(Jane Austen House, 2022) Tokenising the version used for this project reveals that it is a little shorter at 23,886. The text also has approx. 3784 unique words.
The model will be constructed using string sequences of 24 characters with the intension that the model will predict the 25th word. This can then be repeated using the new string and using n-1 to n words and thus predicting the 26th word. This specific model uses 5 layers including an embedding layer to cover the data so as it can be utilised in subsequent layers effectively. This method has been provided as part of the NLP - Natural Language Processing with Python course by Jose Portilla.
The model with over 300 epochs provides the following results:
Utilising this model on a random 26 word string from the existing text, can yield results such as the following:
In [26]: # run model on random set seed_text = ' '.join(random_seed_text)generate_text(model,tokenizer,seq_len,seed_text=seed_text,num_gen_words=25)
Out [26]:Original text: rend='italic'>was</emph an arrival at the hotel but not its amount their visitors answered for two hack chaises could it be the camberwell seminary no no had ...
AI generated text: he seemed to mean any particulars was the same woman called to your leg as this if she wished it was yet of every houseApplying this to the later stages of the text, returns the following:
In [27]: # extract the 10th set of 25 word string from the end and run modelseed_text = ' '.join(text_sequences[-10])generate_text(model,tokenizer,seq_len,seed_text=seed_text,num_gen_words=25)
Out [27]:Original text: poor mr. hollis lt was impossible not to feel him hardly used to be obliged to stand back in his own house and see the best ...
AI generated text: place it can apply to the greater lady especially and green to engage been constrained to pass off sir,&cdq let her place was even toAlas this highlights that there may be some overfitting in the model as it performed better on a section of the text but struggled to perform and create new text. However this establishes how models such as this can generate new text. More advanced models as well as further text cleaning will return better quality text.
Conclusion
As AI develops, there is no doubt that it will not only shape our general lives but also what we consider as uniquely human. Creative fields such as music, arts and literature have already seen the impact of AI and this is unlikely to stop in the near future.
However, as experts frequently note, AIs are so called for a reason. They rely on existing data to create and human involvement is likely to continue for some time. In addition, creative fields such as art and literature, typically aim to explore social and personal challenges issues that at times may be completely knew. For example, conveying life during lockdown or coming of age in a post social media world. They can also use a myriad of formats. For art, this may include water painting, sculpturing and other mediums. For literature, it may be the use of an unreliable narrator, use of second person or using different formats in one tale to convey a story.
This is not to say that AI will not pose a challenge to artists. The capitalist mentality of profits for agencies is sadly an issue that will shape the future of the creative sectors. That said, it is likely to do so regardless of the developments in AI. For example .
However, the notion that AI would completely strip humans form such sectors may still be a notion for a distance future which may never arrive.
“Imagination is everything. It is the preview of life's coming attractions.” ― Albert Einstein
コメント