Content
In November 2022, OpenAI launched ChatGPT, sparking significant interest in the field of artificial intelligence. ChatGPT is a text-based chatbot designed to understand and generate human-like text based on natural language inputs.
ChatGPT, along with other prominent chatbots like Gemini, Claude, Mistral, LLaMA, and DeepSeek, leverages Large Language Models (LLMs) to process and respond in human language. In this article, we will describe how and why these chatbots work, explore the applications of LLMs, and provide illustrative examples.
Understanding Large Language Models
Before we dive into the nuances of Large Language Models (LLMs), it’s essential to understand the broader context of artificial intelligence (AI).
AI refers to the capability of computational systems to perform tasks that typically require human intelligence, such as understanding and generating human language. Within the field of AI, Machine Learning (ML) is a crucial subfield that employs statistical methods to identify patterns in data and model the relationships between inputs and outputs. The core component of this process is the Machine Learning model, which can be thought of as a mathematical function that takes input, performs computations, and produces a prediction.
Deep Learning (DL), a more advanced subfield of ML, uses sophisticated models and vast amounts of data to identify complex patterns. These models, often referred to as neural networks, contain parameters called weights that determine their size and complexity. The more weights a model has, the larger and potentially more powerful it.
Generative AI, a subfield of Deep Learning, focuses on creating new content, and it is within this domain that Large Language Models reside.
LLMs are deep learning algorithms specifically designed for working with text, capable of generating coherent and contextually relevant responses based on the input they receive.
Term | Think of it as… | What it does | Starter Example |
Machine Learning | Model that learns from examples | Makes simple predictions/decisions | Predicting house prices |
Deep Learning | More advanced ML model | Handles complex inputs (images, speech) | Face recognition in photos |
Generative AI | Creative deep learning model | Generates text/images/etc. | Chatbot like ChatGPT, AI art |
Caption: a comparison table between AI fields
Think of it as…
Model that learns from examples
What it does
Makes simple predictions/decisions
Starter Example
Predicting house prices
Think of it as…
More advanced ML model
What it does
Handles complex inputs (images, speech)
Starter Example
Face recognition in photos
Think of it as…
Creative deep learning model
What it does
Generates text/images/etc.
Starter Example
Chatbot like ChatGPT, AI art
Breaking down LLMs
So, with all that prior knowledge in mind, let’s break down the term ‘Large Language Model’ and figure out what it means. We will start with the word ‘Large’. This refers to the size of the text training dataset. To give you an idea of the scale, GPT-3 (one of the earliest state-of-the-art LLMs) was trained using 570 GB of compressed plain text. This is equivalent to more than 1.14 million books like Harry Potter and the Philosopher’s Stone. Nowadays, LLMs contain even more information. The reason for using such a large amount of text lies in the nature of human language. Human language is a diverse construct with patterns (rules), combinations of words and many ways to express the same thing. Therefore, better language understanding comes with more language examples. You can compare it with the process of learning a foreign language.
Next is the ‘Language’. This term highlights the primary function of LLMs, which is to process input text and generate output text. It’s important to note that while LLMs excel at language tasks, they are not designed to perform mathematical calculations. There are numerous examples online where LLMs struggle with simple arithmetic tasks like addition or multiplication.
Lastly, the ‘Model’. Modern LLMs are based on an algorithm called the Transformer, introduced in the 2017 paper “Attention Is All You Need” by Vaswani et al. The core innovation of this algorithm is the Attention Mechanism, which enables computers to understand the importance and relationships of words in a sequence (or sentence). For example, in the sentence “The animal didn’t cross the street because it was too tired,” the Attention Mechanism helps the model determine that “it” refers to “the animal.”
How LLMs work
While we won’t delve into all the mathematical and technical details of the Transformer algorithm, we’ll highlight some key aspects relevant to you as an LLM user. One crucial point is that LLMs do not use words in the way humans understand them. Instead, they use tokens, which can be words, parts of words, or even punctuation marks. For instance, when an LLM processes the input sentence “I am happy!”, it tokenizes it as [“I”, ” am”, ” happy”, “!”]. Generally, LLMs do not need to know all the words that exist; they use a dictionary of tokens to build words, especially long ones. For example, the word “unbelievable” might be split into the tokens “un,” “believe,” and “able,” which can be found as subparts of many other words.
The set of tokens is then represented using embedding vectors, which are numerical representations of text that preserve their semantic meaning. Computers do not understand words as humans do, but they are proficient with numbers. The Transformer algorithm performs the necessary calculations on these embedding vectors to generate the output. However, the output is not a word but a probability distribution over the dictionary of tokens. In the final step, the token with the highest probability is selected, representing the token that occurs most frequently (in reality, the procedure is a bit more complex, but we will avoid the details for simplicity’s sake) in similar sequences in the training dataset.
If you have ever used ChatGPT, you may have noticed that it generates the answer word by word instead of providing the entire response at once. This behavior is due to the auto-regressive nature of LLMs, where the model predicts the output based on all the previous inputs in the sequence until it reaches the end of the output.
You are currently viewing a placeholder content from YouTube. To access the actual content, click the button below. Please note that doing so will share data with third-party providers.
More InformationDifferent use cases
As previously discussed, Large Language Models are designed to work with textual data. This means they can be used for the following tasks:
- Text generation, including content creation and creative writing.
- Question answering and information retrieval; AI-augmented chatbots and knowledge bases
- Document summarisation
- Text translation
- Code generation
- Anything that uses text as a source of information or communication.
A prominent example of a system that leverages an LLM is Retrieval-Augmented Generation (RAG). In essence, RAG is an LLM that has been augmented with a specific source of information (e.g., internal documentation) that is not used for training the model but is instead used to augment its responses with relevant, up-to-date information. This approach enhances the model’s ability to provide accurate and contextually appropriate responses.
Benefits and limitations
As with any kind of AI system, Large Language Models have their benefits and drawbacks, which is useful to understand before integrating them into your business processes.
One of the greatest advantages of LLMs is their ability to handle a variety of text-related tasks in different domains. Furthermore, LLMs produce human-like output that can be customised to your needs. Integrating LLMs enables us to automate repetitive tasks, speed up content generation, save time and money, and support prototyping and creativity.
However, LLMs are prone to hallucinations. This is when an LLM generates incorrect or fabricated text. This occurs when the LLM predicts the most likely text based on the training data. Additionally, the properties of the training data are important. If a dataset contains biased or flawed data, the LLM may generate biased outputs or make flawed decisions. LLMs still lack human understanding and logic, because they rely on patterns found in huge amounts of textual data. Additionally, deploying LLMs requires sufficient computational resources, such as clusters of GPUs or API providers.
The rise of AI Agents
When discussing LLMs, it is almost impossible to avoid the subject of AI agents nowadays. AI agents represent the next evolution in AI technology, building upon the capabilities of LLMs to create more autonomous and functional systems.
Understanding AI Agents
An AI agent is an autonomous software tool that can perform tasks involving natural text generation and beyond. But why do we need AI agents, and how do they compare to LLMs?
LLMs are capable of text generation and operate based on the knowledge introduced to them during training. For example, if a model was trained in 2024, it wouldn’t be able to answer questions based on facts that emerged in 2025. Moreover, LLMs are designed specifically for text processing, and while they excel at this, their capabilities are limited to generating and understanding text. The idea behind AI agents is to extend the range of capabilities of LLMs by introducing actions (functions) in the workflow, thereby acting as a text input/output interface.
AI Agents in action
Let’s consider a simple example to illustrate the power of AI agents. Imagine you want to plan a holiday in Italy. To do that, you need to compare hotel prices and plan the tour. You visit different websites, create a table of possible options and manually compare them, spending a lot of time on the decision-making process.
A specialized travel AI agent offers an alternative to this. This AI agent contains all the necessary tools to automate the scraping of travel websites, making reservations, and much more besides. All you need to do is open the AI agent’s interface, typically an LLM interface, and type the prompt “Plan a 7-day trip to Italy in June”. The AI agent will autonomously collect all the necessary information and present it to you in a formatted way, saving you time and effort.
The great thing about LLMs within AI agents is that they can self-process your natural text input and understand which tools need to be used to complete the task. This ability to interpret natural language and determine the appropriate actions makes AI agents incredibly powerful and user-friendly.
The backbone of AI Agents: Model Context Protocol (MCP)
When discussing the backbone of AI agents, it is essential to mention the technology known as the Model Context Protocol (MCP). Introduced by Anthropic in November 2024, MCP is considered a USB or connector between LLMs and external tools or data sources. MCP provides a standardized way of connecting APIs, data sources, and tools to LLMs.
It contains three key components:
- MCP Server: This component runs your LLM as a backend system, handling the core processing and generation tasks.
- MCP Client: This serves as an interface, often in the form of a chatbot, allowing users to interact with the AI agent in a natural and intuitive way.
- MCP Host: This is where tool calling, context management, and information extraction take place, enabling the AI agent to perform complex tasks and interact with various systems.
Agent-to-Agent (A2A) Protocol
Google took a further step in agentic AI by introducing the Agent-to-Agent (A2A) protocol. The main idea remains the same as with single AI agents, but the key difference is that multiple AI agents collaborate and use each other’s tools to accomplish tasks.
In our travel example, the airline company had its own AI agent, which our travel AI agent contacted to book flights for our planned trip. This collaboration between agents allows for more efficient and comprehensive task completion, leveraging the specialised capabilities of each agent involved.
The A2A protocol enables AI agents to communicate and work together, creating a network of specialised agents that can handle complex tasks more effectively than a single agent could. This collaborative approach is paving the way for more advanced and capable AI systems that can tackle a wide range of real-world problems.
SAP Solutions
SAP Business Technology Platform (SAP BTP)
The primary access point to LLM services within the SAP ecosystem is the SAP Business Technology Platform (SAP BTP). At the heart of SAP’s AI capabilities lies SAP AI Core, a robust platform that enables users to set up various AI tools and access generative AI services.
Through SAP AI Core, users can connect to a range of LLM providers, including industry leaders such as Google, OpenAI, Azure, and Mistral. Additionally, SAP offers self-hosted open-source options for those who prefer to maintain their own infrastructure. For organisations lacking the necessary infrastructure, SAP AI Core provides the capability to host open-source models directly.
Complementing SAP AI Core is the SAP AI Launchpad, a comprehensive management tool designed to streamline the configuration and deployment of LLMs. With SAP AI Launchpad, users can easily:
- Configure their LLM for optimal performance
- Connect to other SAP services for seamless integration
- Monitor resource usage to ensure efficient operation
- Integrate datasets to enhance the LLM’s knowledge and capabilities
- Implement robust security measures to protect sensitive data
SAP Joule: The AI copilot
In May 2024, SAP introduced its own AI system, Joule, marking a significant advancement in the company’s AI offerings. SAP Joule serves as an AI copilot with seamless access to SAP systems, including a variety of AI agents that automate routine business process-related tasks.
One of the standout features of Joule is the Joule Studio, which empowers users to build their own AI agents tailored to their specific business requirements. With Joule Studio, users can
- connect their tools and data sources to create customized AI agents
- deploy these agents using the SAP Cloud infrastructure for scalable and reliable operation
- automate a wide range of business functions, including planning in logistics, accounting configuration, HR management, and more
The versatility of Joule extends to various business functions, including planning in logistics, accounting configuration, HR management, and more. By automating routine tasks and providing intelligent assistance, Joule enables employees to focus on higher-value work and strategic decision-making.
Alongside SAP Joule, SAP also introduced an integrated development environment called SAP Build Code. This environment allows developers to use SAP Joule to create SAP-specific applications and use existing SAP integrations, APIs, and services. Currently, applications can be developed using
- SAP Build low-code for rapid application development with minimal coding
- SAP Build Code for Java or JavaScript for more complex and customized applications
- ABAP for developing applications within the SAP ecosystem
This flexible and comprehensive development ecosystem enables businesses to create tailored solutions that leverage the power of AI and integrate seamlessly with their existing SAP systems.
Demo Video
AI in SAP S/4HANA
Artificial intelligence can now be found in a wide range of applications and is also becoming increasingly embedded in SAP S/4HANA.
In this video, we show you where AI applications are planned or already integrated in SAP S/4HANA. Concrete examples illustrate the added value this brings to your daily work.
PIKON´s solution & services
At PIKON, we specialise in helping businesses integrate Large Language Models (LLMs) and AI agents into their processes. Our comprehensive range of services includes:
- LLM configuration
- Model Selection: Assisting in choosing the most suitable LLM for your specific needs.
- Model Setup: Configuring the selected model for optimal performance.
- Model Fine-Tuning: Adjusting the model to better align with your business requirements.
- Prompt Engineering: Crafting effective prompts to maximize the model’s output quality.
- Model integration
- Configuring Model Access: Setting up secure and efficient access to LLM.
- System Connection: Integrating LLMs with existing systems, such as chatbots, to enhance their capabilities.
- AI Agent Development: Creating customized AI agents tailored to your business processes.
- Workshops and Training
- Explore AI Opportunities: A specialised workshop to identify use cases in your business processes where AI tools can be integrated
- Educational Workshops: Conducting sessions to educate your team about LLMs and the technologies that support them.
- Training Programs: Providing in-depth training to ensure your team can effectively use and manage LLMs and AI agents.
At PIKON, we support our customers in leveraging AI technologies such as LLMs and intelligent agents to enhance their business processes. Our expertise goes beyond AI – we bring deep programming skills, SAP business knowledge, and process know-how to the table. We do not just implement AI for its own sake; we critically assess whether AI is the right solution for a given challenge or if alternative approaches may be more effective, ensuring that technology serves real business value.
Conclusion
The rapid advancement of Large Language Models and AI agents is transforming the way we interact with technology and conduct business. From the foundational concepts of AI and machine learning to the cutting-edge capabilities of LLMs and AI agents, these technologies are opening up new possibilities for automation, creativity, and efficiency. As we have explored in this article, LLMs excel at processing and generating human-like text, enabling a wide range of applications from content creation to customer service. However, they also have limitations, such as the potential for hallucinations and the need for significant computational resources.
Enterprise solutions like SAP are integrating these AI technologies into their platforms, providing businesses with powerful tools for enhancing productivity and driving innovation. With the SAP Business Technology Platform, SAP AI Core, and SAP Joule, organisations can leverage the power of LLMs and AI agents to automate routine tasks, gain intelligent assistance, and create tailored applications that meet their specific needs.
At PIKON, we are committed to helping businesses navigate the complex landscape of AI integration, providing expert guidance and support at every stage of the journey. From LLM configuration and model integration to workshops and training, our comprehensive range of services is designed to empower organisations to harness the full potential of these transformative technologies.
The journey of AI integration is just beginning, and the possibilities are vast and exciting. With the right approach and expert guidance, organisations can harness the power of these advanced technologies to transform their operations and achieve their goals.
Contact
