Reporting by Vincent Desrochers, SAP Software developer at Hydro-Québec
At the SAPinsider Vegas 2024 conference in March, the SAPinsider Developers Challenge took place. This was the first developer’s challenge at this conference since COVID, and four brave teams showed up and delivered various solutions in less than eight hours, giving participants expert guidance from the SAP Developer Advocate team.
Before I entertain you with my experience with this team, allow me to make a parallel between the emergence of GenAI in the SAP world and this year’s total eclipse in North America. The new moon of GenAI covers the “old” sun of hand-made pro-code – only to reveal a shiny new GenAI sun shining and making software development accessible to more people.
The challenge itself
Using the new SAP Business Technology Platform (BTP) SAP AI Launchpad service as the main BTP service, we were asked to develop an application using the AI capabilities of this service in just eight short hours. We then demonstrated the application to a jury decided by SAP and SAPinsider professionals who would crown the winner.
AI Buzzwords
Foundation models
Foundation models are a form of generative artificial intelligence. They generate an output from one or more inputs (prompts) in the form of human language instructions.
They are large machine learning models trained on huge amounts of unlabelled data, usually by supervised self-learning. These generalized models can perform a wide variety of tasks, such as image classification, natural language processing and answering questions, with remarkable accuracy.
GAN (Generative Adversarial Networks) :
A GAN consists of two neural networks competing to become the best. The first network, called the “generator”, will create samples. The second, called the discriminator, will try to detect if a sample is created by the generator or is a real sample for an existing sample library.
LLM (Large Language Models) :
LLMs are used for natural language processing, text generation and answering questions. Examples include ChatGPT, BERT and OpenAI’s GPT-n series. They are deep learning algorithms that understand and generate human language using a large amount of data.
VAE (Variational Autoencoders) :
VAEs represent another type of generative model that leverages the principles of statistical inference. They work by encoding input data into a latent space (a compressed representation of the data) and then decoding this latent representation to generate new data.
Multimodal models:
These combine multiple modalities (text, image, audio) for complex tasks such as image description.
Retrieval Augmented Generation (RAG):
RAG is an advanced artificial intelligence (AI) technique that combines information retrieval with text generation. It allows AI models to retrieve relevant information from a knowledge source and incorporate it into the generated text.
In other words, it enhances the quality of the responses generated by language models by anchoring them on external sources.
AI agents:
AI Agents are computer programs or systems designed to perform tasks autonomously, just like a human agent would.
These digital assistants interact with data, make decisions based on predefined rules, algorithms and machine learning. They act as intermediaries between users and complex systems, simplifying interactions and improving the user experience.
Hallucination:
Hallucination in AI refers to an erroneous response produced by a language model where inaccurate information is presented as true. In other words, AI can “hallucinate” answers that aren’t based on real data or don’t follow any identifiable pattern. This can have significant consequences in a real-world application, such as the spread of misinformation or medical errors.
SAP Service used in BTP
SAP AI Launchpad
SAP AI Lauchpad is a SaaS application from SAP BTP. Customers and partners can use SAP AI Launchpad to manage AI use cases (scenarios) across multiple instances of AI runtimes (e.g., SAP AI Core). SAP AI Launchpad also provides generative AI capabilities through the Generative AI HUB.
Here’s an architectural view of SAP AI Launchpad
Command Prompt Engineering – some advice
The main technique that we used in this challenge was prompt engineering. Here’s some advice on how to make prompts as specific as possible and be creative to try different prompts or phrases during use.
Some techniques include:
• Zero Shot Prompt: Example translating this phrase “Hello” into French
• Prompt for a few shots: Give examples and then ask for answers.
• Go to step (chaining): Step 1 First do this Step 2 then do that, That was the technique we use in this challenge
• More complex techniques:
Command Prompt Configuration
Parameters
Frequency Penalty: Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model’s likelihood to repeat the same line verbatim.
Presence Penalty: Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model’s likelihood to talk about new topics.
Max-Tokens: The maximum number of tokens allowed for the generated answer.
Temperature: What sampling temperature to use, between 0 and 2. Higher values will make the output more random, while lower values will make it more focused and deterministic.
Models available in SAP AI Launchpad
Depending on speed and quality of data required we had to choose a model. In our case we chose GPT-4 to obtain the best quality of coding.
How to do the configuration:
Prompt LLMs in the generative AI hub in SAP AI Core & Launchpad
My developer challenge example:
I utilised my employer, Hydro-Québec, a public utility company owned by the providence of Quebec that generates, transports and distributes electricity to the entire province. Close to 100 percent of the electricity Hydro-Québec distributed to its customers is generated from renewable resources.
To improve the dispatch efficiency of Hydro-Québec’s Montreal-based repair team, we enabled GenAI with the following steps:
1. Generate 6 incidents (power outage) with following fields:
-cause, GPS coordinates, customer name, customer address, priority (1-5, 5 being the highest)
2. Generate 3 repair teams:
– team number, team name, and GPS coordinates
3. Match the 3 repair teams to the 3 incidents with the highest priority
4. Generate some JavaScript to display results
Step 1 – Generate Power outage incidents in JSON
The prompt: Write at least 6 power outages in JSON format with the cause of the outage, with a priority between 1 and 5 depending on the severity of the outage, an actual GPS coordinate, and an existing corresponding address in Montreal, Quebec in the month of March 2024 for Hydro-Québec. Associate a customer name, customer number, and customer phone number with the outage. Only move on to the next step when you’ve written 6 different faults.
Please note that all original prompts were in French and asking for French data, meaning that the Model supports other languages than English, but for ease of reading it has been translated to English.
Step 2 – JSON Repair Teams
Prompt: “Writes at least 3 different repair teams in JSON format in Montreal, Quebec with a team name and its GPS coordinates. Don’t move on to the next step until you’ve written 3 different teams”
Response :
Step 3 – Associate incidents to crews
Prompt : “Assign each of the 3 repairs crew from step 2 to an incident from step 1. Assign the incidents based on highest priority and display the result in Json format. Do not use python code, just display the result in Json”
Response:
As you can see the AI was able to combine the first two steps and assign the incident with the highest priority to the three teams. This may not seem like much, but it was to prove that business rules can be introduced when combining data.
Step 4 – Display program
Prompt > Write a JavaScript program embedded in an html page that will display the answer to step 3.
Response:
Each iteration gave a slightly different program but essentially it was able to take the JSON data from step 3 and incorporate it into the JavaScript code.
Program Output
Lessons Learnt
1.Prompt Engineering is a process and a practical technical skill. Start small and add details and steps as you go. Don’t be afraid to change your approach, as GenAI is doing the heavy lifting.
2. AI can be like a child; you must be very specific but sometimes it will not listen to you. You have to instruct it step by step if need be.
3. Generated code is sometimes prone to hallucination. Always review the code.
4. Sometimes the GenAI gets lazy, it will ask you to finish the job. For example It may generate two incidents and for the remaining four It will use the phrase “… //and so on”.
This challenge at SAPinsider 2024 proved that GenAI combined with proper prompt engineering is a very powerful tool. One must master the different prompting techniques, use the right models and set-up the right values for the parameters to get a result that can be used in real-life scenarios. Fortunately, services like the SAP AI Launchpad and SAP AI Core allow prompts and interface easily with GenAI. With practice and reading about the techniques it can be achieved.