Introduction to Forward Deployed Engineering and GenAI Foundations

This is the first note of the Forward Deployed Engineer (FDE) series. It has two halves. The first half is about the job: what an FDE does and how to turn a vague business request into a real engineering problem. The second half is a short history of AI, from hand written rules to GPT. After it, the model we build around will not feel like a black box (a thing you cannot see inside).

1. From a Business Problem to a Real Engineering Solution

1.1 The situation

Picture a big e-commerce company. It gets about 50,000 customer support requests every day. The messages look like this:

• "Where is my order?"
• "I received the wrong product."
• "Can I return an item after 15 days?"
• "My payment was deducted, but the order was not created."
• "Please cancel my order and start a refund."
• "The delivery agent marked my order as delivered, but I never got it."

The company comes to us and says:

"We want an AI chatbot that can answer customer questions."

That sounds like a requirement. From an engineering point of view it tells us almost nothing. So the first question is always the same:

What problem is the company actually trying to solve?

1.2 Finding the real requirement

The company does not really care about having a chatbot. It cares about things like:

• Customers waiting too long for a reply
• Support staff answering the same questions again and again
• Different staff giving different answers to the same question
• Support cost going up as the company grows
• Important cases getting buried under routine ones
• One customer having to contact several departments for one problem
• No support at night or on weekends

So a much better requirement is:

Reduce the time it takes to resolve a customer's problem, without reducing accuracy or the customer's trust.

That is very different from "build an AI chatbot". The chatbot is just one possible solution.

1.3 Attempt 1: rule based support

Before touching AI we could build an old style rule based bot:

IF message contains "order status" THEN show order tracking page
IF message contains "refund"       THEN show refund policy
IF message contains "cancel"       THEN show cancellation instructions

This works when the input is predictable. Humans are not predictable. Four customers with the same problem might write:

• "Where is my order?"
• "It has been seven days and my package has not arrived."
• "Bhai mera parcel abhi tak nahi aaya."
• "The expected delivery date has passed. Could you check what happened?"

Same intention, completely different words. You could keep adding rules, but the list of things to handle never ends: thousands of ways to say one thing, rules that clash with each other, spelling mistakes, many languages, Hinglish, context from earlier messages, two problems in one message, and situations nobody thought of.

The root issue: human language has too many variations to describe with hand written rules.

1.4 Attempt 2: machine learning classification

Instead of writing every rule, train a model on examples:

The model learns the pattern and predicts a category for a new message. This is better than keyword matching, but it only solves a small slice of the problem. Knowing that a message is about a refund does not:

• explain the company's refund policy
• find the customer's order
• check if the refund is allowed
• ask for missing details
• actually start the refund
• handle odd cases
• escalate to a human
• explain why a request was accepted or rejected

Classification is one step in the workflow. It is not the whole business problem.

1.5 Attempt 3: a general purpose LLM

A Large Language Model (LLM) is a program trained on huge amounts of text to write text. It is far more flexible. It can understand a request, summarise a complaint, detect intent (what the customer wants), pull out an order id, translate, write a reply, ask follow up questions, read a policy you give it, and decide which tool might help.

But look at this:

System:   You are a customer-support assistant. Answer the customer's question.
Customer: Can I return a laptop after 20 days?
Model:    Yes, laptops can be returned within 30 days of delivery.

Sounds professional and confident. Now suppose the company's real policy says laptops can be returned only within 7 days. The answer is completely wrong.

This is the first big lesson of the series:

Producing a convincing answer and producing a correct business answer are two different problems.

1.6 An LLM does not know your company

A general LLM knows what refunds, cancellations, payments and deliveries mean in general. It has no idea about:

• your current refund policy
• this customer's order history
• the live delivery status
• payment records, inventory, internal procedures
• policy changes made last week
• who this customer is and what they are allowed to do

So the application has to give the model the right information. An answer built on that information is called grounded, because it rests on real facts:

The LLM is just one component of the application.

1.7 Generating text is not the same as doing something

A customer says: "Cancel my order and refund my payment."

The model can easily write "Your order has been cancelled." Writing that sentence cancels nothing. A real system looks more like:

And the moment you start building this, engineering questions pile up:

• Is this user allowed to cancel this order?
• Has the product already shipped? Is this order even cancellable?
• Should the LLM call the cancellation service directly?
• Should large refunds need a human's approval?
• What happens when the API fails?
• What if the model picks the wrong order id?
• How do we audit everything that happened?
• How do we test this? What does each conversation cost? What latency is ok?
• When should it hand over to a human?

So the original request, "build an AI chatbot", has turned into:

Build a secure and reliable system that combines an AI model, company knowledge, customer data, existing APIs, validation rules and human oversight to resolve customer support requests.

The gap between those two sentences is exactly where a Forward Deployed Engineer earns their keep.

2. What Is a Forward Deployed Engineer?

Take the name apart.

Deployed does not mean "deploy an app to a server". It means the engineer is placed close to where the real problem lives. An FDE works directly with customers, clients, business departments, operations teams and the customer's existing engineering teams. They watch the real workflow, understand the constraints, build the solution and help get it into real production use.

Forward is about position. A normal product team builds a general platform that serves many customers. An FDE sits much closer to one specific customer's problem.

The FDE works at the boundary between the technology and the customer's actual work.

Definition. A Forward Deployed Engineer is a software engineer who works closely with customers or operational teams to understand important problems, builds technical solutions around their real systems, and takes those solutions into production (live use by real users).

The role mixes several skills: software engineering, product thinking, system design, integrations, quick experiments, production delivery, evaluation and troubleshooting.

One more thing makes the job different. The customer often does not know what technical solution they need. Working that out is part of the job.

2.1 Request vs requirement

"We need an AI chatbot" is a proposed solution. The engineer has to find the requirement behind it. The mindset for the whole series:

2.2 Goal of the series

By the end, the goal is to take a vague request like "build an AI assistant for our support department" and turn it into a working engineering solution built on generative AI. Before we can do that, we need to understand the technology we are building around. So the rest of this note is about AI itself.

3. Understanding Artificial Intelligence

3.1 Where AI began

The question behind AI is simple: can a machine do things that seem to need human intelligence?

Early computers did calculations and followed exact instructions:

Input: 20 + 30
Instruction: add the numbers
Output: 50

Programming languages gave us variables, conditions, loops, functions, recursion, data structures and algorithms. All of these let a human describe steps for a computer to execute. Traditional programs work brilliantly when the input is clear, the operation can be defined exactly, the expected result is known, and every step can be written as an instruction.

Human intelligence is messier. Humans understand language, recognise objects, solve unfamiliar problems, learn from experience, decide with incomplete information, explain ideas, and create stories, images, music and programs. That raises a deeper question: can thinking itself be written down as exact steps that a computer can run?

3.2 Alan Turing and the question of machine intelligence

In 1950 Alan Turing published Computing Machinery and Intelligence. It opened with: "Can machines think?"

The trouble is that "thinking" is hard to define even for humans. We cannot see someone's thoughts. We see behaviour and infer intelligence. So Turing swapped the question for something you can observe:

Can a machine communicate convincingly enough that a human judge cannot reliably tell it apart from another human?

This became known as the Turing Test:

Communication is by text, and the judge does not know which side is the machine. The point of the test was not a recipe for building AI. It moved the conversation from philosophy to observable behaviour. If a machine can understand questions, keep context, give relevant answers, explain ideas and handle unfamiliar situations, it is showing behaviour we associate with intelligence.

One warning that matters a lot for modern LLMs: sounding intelligent is not the same as being correct.

3.3 The Dartmouth workshop and the term "Artificial Intelligence"

In 1955 a group of researchers proposed a workshop, held at Dartmouth College in 1956. The proposal used the phrase Artificial Intelligence and helped make AI a research field of its own. The idea: if we can describe a part of intelligence exactly, a machine may be able to copy it. They explored language, problem solving, forming abstractions (general ideas) and learning from experience.

So the dream is old. What changed over the following decades is how strong our systems can be. Better algorithms, more data and far more computing power made the difference.

3.4 AI is much bigger than LLMs

AI covers chess programs, medical diagnosis systems, face recognition, recommendation engines, self driving, speech recognition, language models and image generation. So:

Artificial Intelligence ≠ Large Language Models.

An LLM is one kind of AI system. Generative AI is one big area inside the much larger field of AI.

4. Rule Based AI

One early approach: if intelligent behaviour follows rules, maybe humans can write those rules down.

IF age >= 18 THEN eligible ELSE not eligible

Age 17 gives "not eligible", 18 and 25 give "eligible". Fully deterministic: the same input always gives the same output. A troubleshooting rule works the same way:

IF device does not turn on AND battery level = 0
THEN recommend charging the device

Here the human writes the knowledge down directly. This is usually called symbolic AI (the knowledge is kept as written symbols and rules).

4.1 Parts of a rule based system

1. Facts describe the current situation. "Customer has a premium account. Order was delivered 3 days ago. Product is returnable. Product is damaged."
2. Rules say what should happen. "IF product is damaged AND delivered within 7 days AND returnable THEN customer is eligible for replacement."
3. Inference combines the facts with the rules to reach a result. Here it means reasoning with rules. It is not the "inference" of section 5.7, which means running a trained model.

The computer is not inventing policy. It is applying logic a human wrote.

4.2 The natural language problem

The business rule can be tiny:

IF daysSinceDelivery <= 7 THEN returnEligible = true

But customers do not send {"daysSinceDelivery": 4}. They send "I received this some time last week. Can I send it back?" Now the system has to work out what "this" is, what date "last week" means, which order they mean, whether they want a refund or a replacement, and whether the product is returnable at all.

The rule is easy. Turning human language into the facts the rule needs is the hard part.

4.3 Early chatbots and expert systems

Early research went in two directions:

• Chatbots: can a machine talk like a person?
• Expert systems: can a machine decide like a specialist?

4.4 ELIZA

ELIZA was an early program that simulated conversation:

User:  I am feeling unhappy.
ELIZA: Why are you feeling unhappy?
User:  My friends do not understand me.
ELIZA: Why do you think your friends do not understand you?
User:  They never listen to me.
ELIZA: Can you tell me more about them?

It feels meaningful, but underneath it is patterns and templates:

IF user says "I am <something>"   RESPOND "Why are you <something>?"
IF user mentions "mother"          RESPOND "Tell me more about your family."

Swap a few pronouns around and the reply looks like a real conversation.

4.5 The illusion of understanding

ELIZA taught us something about people, not just machines: a system can produce language that feels meaningful without understanding anything. Humans naturally assume meaning and intention behind words. If a program asks "How does that make you feel?", we assume it knows what "that" is. It may just have picked a template.

Modern LLMs are vastly more capable and are not template lookups. But the lesson stays: fluent language is not proof of factual reliability. A fluent system tempts users to assume it knows what is true, understands the world like a human, checked its answer, remembers things it was never told, and really did the action it described. Every one of those assumptions is dangerous in a real application.

4.6 Why hand written rules do not scale

Say support needs to handle only ten intentions: order tracking, cancellation, refund, replacement, payment failure, damaged product, missing item, account problem, delivery complaint, product information.

Look at order tracking alone:

• Where is my order?
• Track my package.
• My parcel has not arrived.
• Delivery kab hogi?
• Order abhi tak nahi mila.
• The expected date has passed.
• Can you check shipment ORD-123?
• Your app says dispatched, but nothing arrived.

Now add typos, abbreviations, sarcasm, other languages, Hinglish, missing details, several issues in one message, earlier conversation context, different policies per region, and policies that change. The number of combinations explodes.

That pushed researchers to a new idea: instead of writing every rule, can the machine learn the patterns from examples?

5. Machine Learning

5.1 Learning patterns instead of writing rules

How do you teach a child what a dog is? You could try rules: four legs, fur, two ears, a tail. Those describe a lot of other animals too. Add "barks", "a particular face", "particular proportions", and then remember a Chihuahua and a Great Dane look nothing alike.

Children actually learn from examples. "This is a dog. This is also a dog. This is a cat. This is another dog." After enough examples they can tell dogs apart without ever getting a mathematical definition.

Machine learning follows the same idea: give examples, let the system learn the patterns that connect inputs to outputs.

5.2 Traditional programming vs machine learning

In normal programming, humans provide the rules. In machine learning, humans provide examples, a goal for the learning, the shape of the model (its architecture) and the training process. The model then adjusts its internal numbers, called parameters, to find useful patterns.

5.3 Example: spam detection

A rule like IF email contains "win money" THEN spam breaks immediately. A genuine email can contain those words, and a spammer just changes the wording. A machine learning system instead learns from many examples of email -> spam and email -> not spam, picking up patterns in the data (such as which words often appear in spam) rather than one hand written rule.

5.4 Unsupervised learning

Not every problem comes with labelled examples. Suppose you have a million support messages and nobody has categorised them. A system can still find groups on its own:

Group 1 -> delivery related conversations
Group 2 -> payment related conversations
Group 3 -> product quality complaints

It discovers structure in the data without being told the right category for each message. That is unsupervised learning.

5.5 Machine learning does not learn "truth"

A model learns patterns from whatever data it is given. If that data is wrong, incomplete, biased, outdated or unrepresentative, the model learns and repeats those problems. Machine learning is not a magic route to objective truth.

5.6 What is a model?

At its simplest, a model is a computational system that maps an input to an output.

A token is a small piece of text, such as a word or part of a word.

5.7 Training vs inference

Training: the model sees many examples, produces outputs, those outputs are scored against the goal of the training (how close they are to the right answer), the parameters are adjusted, and this repeats many times. Modern models have billions of learned parameters and training them takes enormous computation.

Inference: once trained, the model is used on new input. For an LLM, that is prompt in, generated response out. When an application calls an LLM through Spring AI, Python, JavaScript or a plain HTTP call, it is doing inference.

5.8 Does the model learn from every prompt?

No. A prompt is temporary input. It shapes the current response, but a normal inference request does not change the model's trained parameters.

Putting information in a prompt and training the model are two different things.

6. From Machine Learning to Neural Networks

6.1 Feature engineering

Older machine learning depended on feature engineering. A feature is one clue the model looks at. Take face recognition. The computer does not see a person, eyes, nose or mouth. It sees pixel values. So humans designed features: edges, shapes, colour differences, eye like patterns, symmetry, distances between facial parts. For text, humans designed features like word frequency, keywords, sentence length, punctuation and grammar tags.

Deciding by hand which characteristics the model should look at is feature engineering.

6.2 Why it became a bottleneck

1. It took a lot of human effort, per problem.
2. Humans miss patterns. The most useful signals are often ones nobody would think to write down.
3. Features did not transfer. Face features are useless for dog recognition, speech or language.

Which leads to the real question: what if the useful pattern is too complicated for a human to describe at all? Neural networks are one answer.

6.3 Building the idea of a neuron

Say we want to predict whether a support ticket is urgent. Inputs might be:

x1 = contains the word "urgent"
x2 = payment failed
x3 = customer contacted support several times
x4 = customer is locked out of their account

Not every input matters equally. Being locked out probably matters more than the word "urgent". So the model gives each input a number called a weight, which says how much that input counts.

The unit adds up the weighted inputs, and an activation step turns the total into an output. It is loosely inspired by a brain cell (a neuron). One unit can only model simple relationships, so we connect many of them.

6.4 Neural networks

A neural network is layers of these units connected together:

The layers between input and output are called hidden layers. During training the network adjusts a huge number of weights, so it can learn complicated links between the input and the expected output.

6.5 Representation learning

Deep learning means neural networks with many layers. Its big strength is representation learning. Older approaches needed humans to decide how to describe the data to the model. Deep networks learn their own internal description, a set of numbers, directly from data.

These four sentences use different words but mean nearly the same thing:

• Where is my parcel?
• Track my package.
• My delivery has not arrived.
• Order abhi tak nahi aaya.

A network can learn numbers for each sentence that put these four close together. It does not store something like ORDER_TRACKING = these exact sentences. The information is spread across many learned numbers.

6.6 AI, machine learning and deep learning

These are nested, not separate:

The goal of this series is not to become deep learning researchers. We need enough understanding to build reliable applications around these systems.

6.7 Neural networks and computer vision

Neural networks first had huge success in vision. Convolutional Neural Networks (CNNs) learned visual patterns such as edges, shapes, textures, objects and faces, and powered facial recognition, photo tagging, object detection and self driving. Nobody had to hand specify every visual feature any more.

Language turned out to be a different kind of problem.

7. Why Human Language Is Hard for Computers

Language feels natural to us because we spend our whole lives using it. For a machine, the meaning of a sentence is almost never decided by the individual words alone. It depends on context, word order, grammar, the speaker's intention, shared knowledge, culture, earlier conversation, tone and ambiguity.

A useful language system has to learn patterns that still work on new combinations of words and ideas.

8. The Evolution of Language Models

A language model estimates which piece of text is likely to come next, given the text so far.

The sun rises in the ___        -> "east" is very likely
She deposited the money in the ___  -> "bank" is very likely

The key idea: how likely the next piece of text is depends on the context before it.

8.1 Early statistical language models

Before neural networks, one approach was simply counting how often words appear together. With this training text:

I like machine learning.
I like Java programming.
Students like machine learning.

After the word "like" we saw "machine" twice and "Java" once, so:

P(machine | like) = 2/3
P(Java    | like) = 1/3

P(machine | like) reads as "the probability of machine, given the word like". Given "I like ___", "machine" gets the higher probability. That is a basic statistical language model.

8.2 N-grams

An n-gram is a sequence of n items, usually words.

A bigram model uses P(next word | previous word). A trigram model uses the previous two words. Either way, the model only looks at a small fixed window of context.

8.3 Why context matters

I went to the bank to deposit some ___

If the model only sees "some", it knows almost nothing. Watch what happens as context grows: "bank", "the bank", "went to the bank", "went to the bank to deposit some". Each step makes "money" or "cash" easier to guess.

The meaning and the likely continuation of text depend heavily on context.

8.4 Problems with n-gram models

Suppose the model saw "The customer requested a refund." but never saw "The buyer asked for reimbursement." A pure counting system treats those as completely unrelated, even though a human instantly sees customer ≈ buyer, requested ≈ asked for, refund ≈ reimbursement.

On top of that, language has an enormous number of possible word combinations, and most of them appear rarely or never in the training data. So the counts are mostly empty. This is the sparsity problem. Models needed a better way to learn relationships between words and context.

8.5 Recurrent Neural Networks (RNNs)

Neural networks could hold richer information about words. Early neural language models used RNNs, which read text one word at a time and keep an internal state (a running memory):

The state tries to hold on to whatever was useful from earlier words. Reading sequentially makes sense, since that is how language arrives and it preserves word order. The trouble starts when the sequence gets long.

8.6 The information bottleneck

An RNN keeps squeezing everything it has seen into one evolving state. Imagine reading a 20 page policy one word at a time while keeping a tiny set of notes that has to hold everything important. As the text grows, earlier information fades, details get lost, useless words take up space, and links between far apart words become hard to keep.

The replacement policy for laptops purchased during the festival sale, which
was updated after several suppliers changed their warranty terms, does not
apply to refurbished devices.

To understand "does not apply", the model has to connect it back to "The replacement policy" and forward to "refurbished devices", with a pile of unrelated words in between.

8.7 Long range dependencies

A long range dependency is when understanding one part of a sequence needs information from a distant part.

The laptop that the customer ordered after comparing several models from
different brands was damaged.

What was damaged? The laptop. The model must hold that link across many intermediate words. Handling this well became one of the main challenges in language modelling.

8.8 Attention

Start with the problem it solves:

Riya gave Priya the laptop because she needed it for work.

To understand "she", the model has to decide between Riya and Priya. To understand "it", it has to connect back to "laptop". Different words need information from different places in the sentence. So the model needs a way for one position to look at other relevant positions. That is attention.

While processing a piece of text, the model effectively asks: which other parts of the context are useful for understanding this part right now?

The animal did not cross the street because it was tired.

For "it", "animal" is more relevant than "street", because an animal can be tired and a street cannot.

Nobody programmed IF word = "it" THEN look for animal. These attention patterns are learned in training.

Attention is context dependent. For "bank" in "I deposited money in the bank", the useful neighbours are "deposited" and "money". For "bank" in "We sat on the bank of the river", they are "sat" and "river". Same word, different context, different meaning, and attention is the mechanism that pulls in the right context.

8.9 The Transformer breakthrough

A Transformer is a network design that puts attention at the centre, instead of one running state like an RNN. That fixed two limits of the older sequential models: one state that squeezed everything and let early details fade (8.6), and the struggle to link far apart words (8.7). It became the foundation of modern LLMs. ChatGPT, Gemini, Grok and most other current language models are built on Transformer architectures.

8.10 Attention is not intelligence

Attention is, in the end, maths for finding and combining relevant information. On its own it does not mean the model knows whether a statement is true, understands like a human, has current information, can see your private company data, can act in the real world, or always decides well. Keep this in mind whenever you build on top of an LLM.

9. Welcome to Modern Large Language Models

Modern LLMs combine large neural networks, huge training datasets, learned numerical representations, Transformer architectures, attention, and a lot of computing power. The most recognisable name is GPT.

Autoregressive means the model writes one token at a time, and each new token is added to the input for the next step.

10. The Complete Journey So Far

Each generation fixed a limitation of the previous one:

11. Back to Forward Deployed Engineering

Why bother with all this history? Because an FDE must not treat the model as magic. An LLM is a strong component, but a production system still needs engineering around it.

The LLM handles language and tasks that look like reasoning. The application around it makes the final system useful, grounded, secure, reliable, testable and observable (you can see what it is doing). It also connects the system to real tools and lets it take real actions. Building that complete system is the engineering problem this series is about.

12. Key Takeaways

1. A customer's proposed solution is not always the real requirement.
2. FDEs work close to real customer problems and turn business needs into production ready technical solutions.
3. AI is a much broader field than generative AI or LLMs.
4. Rule based systems depend on humans writing the knowledge and logic down.
5. Human language is too varied and context dependent for hand written rules.
6. Machine learning lets a system learn patterns from examples.
7. A model is a computational system that maps inputs to outputs using learned parameters.
8. Training changes the parameters. Normal inference only uses them.
9. Deep learning learns useful internal representations instead of relying on handcrafted features.
10. Language is hard because meaning depends heavily on context.
11. Statistical language models introduced the idea of predicting likely continuations.
12. RNNs processed language sequentially but struggled with long range relationships.
13. Attention lets each part of a sequence pull information from the relevant parts of the context.
14. Transformers put attention at the centre and became the base of modern LLMs.
15. GPT stands for Generative Pre-trained Transformer.
16. GPT style models generate text by repeated next token prediction.
17. Fluent language does not guarantee factual correctness.
18. An LLM alone is not a complete production application.
19. Real AI systems need data, APIs, business rules, security, validation, evaluation and often human oversight.
20. Building that complete system is where the FDE mindset earns its value.

Next: How Large Language Models Work, where we open the box and follow a sentence from text to tokens to the next predicted token.