Machine learning experts at iteratec have developed a text and image generator for the in-house marketing team. This makes it quick and easy to write champion stories about successful client projects.
Like many other departments, since the launch of ChatGPT we have been asking ourselves how artificial intelligence in general, and large language models in particular, can be used to our advantage. Here, we would like to share the insights we have gained from our project. These insights are relevant to anyone who creates structured texts or reports using generative AI, generates images using generative AI, or wishes to learn about prompt engineering through a real-world use case.
Contents
To develop an intuitive tool that generates text and suitable images for ‘Champion Stories’ – customer projects featured on our website. That was the objective. The ‘Champion Stories’ were an ideal use case, as they are always structured in a consistent way: from the customer’s challenge to the solution. This enabled us to develop a prompt based on precise specifications. The end result was a web interface that first converted an unstructured briefing into a structured one and then used this to generate text and images.
Behind the scenes, the tool utilises the OpenAI API, various image generation APIs such as DALL-E and Stable Diffusion, and has access to stock photos. In addition, ChatGPT and GitHub Copilot were used during the project to harness the full potential of generative AI and gain practical experience.
Prompt engineering was the most important factor in the application’s success. Prompts – that is, instructions given to the AI tool – must be formulated in such a way that the user’s expectations and the result align as closely as possible. Although many tech companies now offer tips on prompt engineering in blogs and documentation, there is usually a lack of clear guidance on how to achieve the desired result. We experimented.
The biggest challenge was to consistently analyse and evaluate the long texts – often spanning two pages – in a thoroughly structured manner. Why? Firstly, minor details in the prompt can have a major impact on the result. This made it difficult to determine which change in the prompt had altered the outcome. Secondly, the output text varies due to a certain degree of randomness, making it difficult to determine whether the result is consistently good or whether the output was simply a lucky exception. Deciding whether a generated text is well-written and makes sense in terms of content takes time and is often subjective.
It can also be assumed that many different prompts lead to good results. Therefore, it is sufficient to find one good prompt. Rapid and iterative testing of different prompt variations is thus far more efficient than a meticulous individual evaluation of every change.
Ultimately, we concluded that the evaluation criteria (e.g., writing style and text length, minimal hallucinations, information content, or consistency) and evaluation methods (e.g. expert assessments, defining a scoring scale, or automated text analysis) should be established and prioritised right from the start. The objectives must also be clearly defined and can be adjusted iteratively as required.
It also helped us to quickly make the Minimum Viable Product (MVP) available in the form of an application. This enabled the team to test as early as possible whether the prompt generated the desired texts. Alternatively, the prompt could also be tested directly in ChatGPT.
OpenAI has provided further information on best practices for prompting here.
Generative AI is still a long way from understanding prompts like a human would and acting accordingly. Instead, the prompt is used to generate the text that is most likely to be appropriate. This probability is based on all the training data – essentially the entire internet. Consequently, the tool struggled to produce longer texts with a predefined structural framework.
For example, repetitions posed a challenge. Much of the content appeared repeatedly across several paragraphs, and the generated text did not deliver the desired content for each paragraph. The reason for this was that the briefing had originally been written for humans, not for ChatGPT, and therefore allowed for a great deal of flexibility. The briefing was therefore worded much more strictly, and word repetitions were avoided.
Furthermore, ChatGPT tends to ignore task specifications, even when the prompt contains an explicit requirement. One example is that our champion stories require three different headings. Despite detailed instructions in the prompt, ChatGPT did not always implement this reliably. The solution was to use two separate prompts: the first prompt is used to create a story with just one headline, and the second prompt is then used to generate further headlines. This led to a significant improvement in text quality.
To create suitable images for the Champion Stories, we used ChatGPT to generate short and concise ideas for images. Users can choose from five options within the application.
The key challenge lay in designing the prompt for ChatGPT to generate high-quality ideas and guidelines for image generation. The generated ideas for image creation often sounded promising and were thematically appropriate. Nevertheless, problems arose during the creation of the images, including various artefacts, errors in faces and in details, and not all specifications were always adhered to. To resolve this issue, we attempted to use the prompts to prevent fine-scale structures in the images, in order to avoid potential artefacts and errors. This led to some improvements, but the resulting images were still clearly recognisable as artificial and incoherent.
In summary, it is possible to generate topic-relevant images and provide various options using a brief project description or a briefing. A GenAI tool such as ChatGPT, when provided with the correct prompt, can supply information to help create suitable prompts for image generation. However, generating images that meet iteratec’s requirements for photorealistic representations – and, ideally, the inclusion of people – remains an extremely challenging task.
Initially, we used OpenAI’s standard API, which meant that the briefing data and information about client projects were stored on US servers. We therefore used only Champion Stories and content that had already been approved for publication for training purposes.
However, one solution was to switch to the Azure OpenAI Service. This service provides access to OpenAI’s GPT-3 models with enhanced enterprise features, including security, compliance and regional availability. By using the Azure OpenAI Service, we can store data locally whilst complying with data protection guidelines, whilst continuing to benefit from its powerful text generation capabilities.
The finished web application, including prompts, has already been successfully used for testing. Although the generated text is not error-free and will be further refined and adapted in the future, the application provides significant support in establishing an initial structure, headings, paragraph content and creative phrasing. The tool significantly speeds up the creation of champion stories. The insights we have gained in the areas of prompt engineering and the integration of ChatGPT form a solid foundation for further projects.
Do you have any questions or need assistance?
You can find out more about the possibilities of generative artificial intelligence for your business on our website. Feel free to get in touch with us.