CodeKerdos.in | Gen-AI Blog Series | Week 2
It’s 11:45 PM. You have a tab open with the OpenAI API docs. Another one with a YouTube video on AI chatbots. And a half-written Python script that, for some reason, keeps giving you answers that are either too vague or completely off-topic.
You have been at this for an hour. You change the question slightly. Different answer, still not what you wanted. You try again. The model goes off in a completely different direction. You start wondering if this thing is even useful, or if everyone else is just pretending it works.
“Why does it work perfectly in the demo videos but keep missing the point for me?”
Here is the truth: the model is not broken. The input you are giving it is just not precise enough. And that gap, between what you type and what you actually mean, is exactly what prompt engineering is designed to close.
This is not about being clever with words. It is a technical skill, one that directly affects the quality, reliability, and cost of every AI-powered feature you build. Once you understand how to write prompts properly, that frustration at 11:45 PM turns into something that actually ships.
Let’s walk through it properly, from the fundamentals to the techniques you will use every day as a developer.
When most developers start using LLMs, they treat the prompt like a search bar. Short, vague, keyword-heavy. They type something like “summarize this document” or “write a function for me” and then wonder why the output is mediocre.
The model has no idea who you are, what you are building, what format you want the output in, what level of detail is appropriate, or what tone fits your audience. When you give it nothing, it fills in all those blanks by guessing. And it guesses based on the average of every piece of text it has ever seen, which is rarely what you need.
Think of it like this. If you walked up to a senior developer on your team and said “write me some code,” they would stop and ask you ten questions before touching a keyboard. What language? What does it need to do? Any edge cases? Does this go to production or is it just a prototype?
A language model will not ask those questions on its own unless you design your prompt to include that context upfront. Your job as the developer is to front-load all of that information so the model can produce something focused and useful the first time around.
Most production-grade prompts share the same building blocks. You do not always need all of them, but knowing what each one does helps you decide when to include it and why.
Telling the model who it is supposed to be is one of the simplest and most effective techniques. It primes the model to draw on a specific area of knowledge and adjust its communication style accordingly.
System: You are a senior Java developer with 10 years of experience in Spring Boot.
You write clean, production-ready code with proper error handling.
When explaining concepts, you assume the reader knows Java basics but may not be familiar with Spring internals.
Notice what that single system prompt does. It sets the expertise level, the style of code, and the assumed knowledge of the audience. Every response in that conversation shifts to match that persona, without you having to repeat yourself.
Context is the background information the model needs to give you a relevant answer. The more accurate context you provide, the less the model has to fill in with guesses.
Context can include: the tech stack you are using, the business problem you are solving, constraints you are working under, decisions that have already been made, and anything else a new team member would need to know before they could help you.
User: I am building a REST API in Spring Boot 3.2 that serves a mobile application.
We use JWT for authentication and PostgreSQL as the database.
Our architecture is layered: Controller, Service, Repository.
I need to implement rate limiting that allows each user 100 requests per minute.
How should I approach this?
This is the core of your prompt. What do you actually want the model to do? Be specific. Use action verbs: explain, generate, list, compare, rewrite, debug, summarize. Avoid vague instructions like “help me with this” or “tell me something about.” Those give the model too much latitude.
If you have multiple tasks, list them explicitly instead of bundling everything into one sentence. The model handles numbered task lists significantly better than run-on instructions.
User: Do the following three things:
If you need output in a specific structure, say so explicitly. Do you want a table? A numbered list? JSON? Markdown? A code block followed by a plain-English explanation? The model will not guess the right format unless you tell it.
User: Respond in the following format:
– Summary: One paragraph explaining the issue.
– Root Cause: One sentence.
– Fix: Code snippet with a brief explanation.
– Prevention: Two bullet points on avoiding this in future.
This matters even more in production applications where you are parsing model output programmatically. If you need JSON, specify the exact schema. If you need Markdown, say so. Do not leave it to chance and then debug why your parser keeps failing.
Providing examples of what you want, and sometimes what you do not want, is one of the most reliable ways to improve output quality. This is called few-shot prompting and we cover it in the next section.
Constraints help you control the scope and shape of the output. They can be about length (keep your response under 300 words), content (do not use technical jargon), tone (write in a friendly, direct tone), or format (respond only with valid JSON, no extra explanation).
Constraints are especially valuable in production. You want predictable, bounded outputs, not open-ended essays that vary wildly from one request to the next.
These terms come up constantly in Gen AI discussions. They are simpler than they sound.
These terms come up constantly in Gen AI discussions. They are simpler than they sound.
You give the model a task with no examples. You are relying entirely on what the model learned during training to produce the right output.
User: Classify the sentiment of the following review as Positive, Neutral, or Negative.
Review: “The app crashes every time I try to upload a photo.”
Zero-shot works well for common, well-understood tasks. It struggles with unusual tasks or when you need output in a very specific or consistent format.
You give the model one example of the input-output pair before the real task. This acts as a template, showing the model exactly what shape you expect the response to take.
User: Classify the sentiment. Use exactly one word: Positive, Neutral, or Negative.
Example:
Review: “The dashboard is clean and easy to use.”
Sentiment: Positive
Now classify this:
Review: “The app crashes every time I try to upload a photo.”
Sentiment:
You give the model two or more examples before the actual task. This uses more tokens, but it is significantly more reliable for tasks where consistency and format matter across many requests.
User: Extract the following fields from each support ticket:
issue_type, urgency (low/medium/high), product_area.
Respond only in JSON.
Ticket: “I cannot log in. Password reset email is not arriving.”
Output: {“issue_type”: “authentication”, “urgency”: “high”, “product_area”: “login”}
Ticket: “The export button on the reports page takes 30 seconds to respond.”
Output: {“issue_type”: “performance”, “urgency”: “medium”, “product_area”: “reports”}
Ticket: “Can you add dark mode to the mobile app?”
Output:
Few-shot prompting is especially powerful for data extraction pipelines, classification features, or any use case where output format must be consistent across thousands of requests. A small investment in good examples saves you hours of post-processing work.
Here is something that surprises most developers the first time they hear it. LLMs produce better answers to complex problems when they are encouraged to work through the reasoning step by step before giving a final answer.
This is called chain-of-thought prompting. Instead of asking for the answer directly, you ask the model to reason through the problem first. The act of generating intermediate reasoning steps actually improves the quality of the final output. It sounds strange, but it holds up consistently in practice.
Without chain-of-thought:
User: A user makes 5 API calls in the first minute, 12 in the second,
and 8 in the third. Is this within a rate limit of 20 calls per minute?
Model: Yes. (sometimes wrong on overlapping windows or edge cases)
With chain-of-thought:
User: A user makes 5 API calls in the first minute, 12 in the second,
and 8 in the third. Is this within a rate limit of 20 calls per minute?
Think through each minute step by step before giving your final answer.
Model: Minute 1: 5 calls. 5 is less than 20, within limit.
Minute 2: 12 calls. 12 is less than 20, within limit.
Minute 3: 8 calls. 8 is less than 20, within limit.
Final answer: Yes, all three minutes are within the rate limit.
The simplest way to trigger chain-of-thought is to add one of these phrases to your prompt:
In production, if you only need the final answer and not the reasoning, you can instruct the model to reason internally and only output the conclusion. Or you can build a two-step pipeline: one pass for reasoning, one pass for the clean answer.
The system prompt is the set of instructions that defines how the model should behave throughout the entire conversation. You, the developer, set this. The end user of your application typically never sees it.
This is where you define the persona, the constraints, the output format expectations, and any other baseline behavior. Think of it as the standing instructions you give to a team member before they get on a call with a customer.
User prompts are the actual messages in the conversation. In a chat application, these come from the end user. In a pipeline, they are often programmatically generated by your application based on data it is processing.
A good architecture keeps the system prompt stable and controlled by you, while the user prompt is dynamic and comes from the outside world. This gives you consistent baseline behavior without locking you into a rigid flow.
Vague vs Specific:
GOOD: Review this Java code for null pointer exception risks and list each risky line with a brief explanation.
AVOID: Review my code.
Jumbled vs Structured:
GOOD: Do the following:
1. List potential bugs in the code below.
2. Rate each bug’s severity as low, medium, or high.
3. Suggest a fix for each one.
AVOID: Can you check my code for bugs and rate them and also fix them and explain why they are bugs?
No Format vs With Format:
GOOD: Extract the customer name, order ID, and issue. Respond ONLY in this JSON format:
{“customer_name”: “”, “order_id”: “”, “issue”: “”}
AVOID: Get the customer name and order details from this support ticket.
Real-world data is messy. You will get missing fields, ambiguous input, and text that does not match the format you expected. Your system prompt should tell the model what to do in those situations instead of letting it improvise.
System: If the ticket does not contain enough information to fill a field, use null.
Do not guess or make up values.
If the input is not a support ticket at all, respond with:
{“error”: “Input does not appear to be a support ticket.”}
Your first prompt is almost never your best prompt. Prompt engineering is an iterative process. You write a prompt, test it on real inputs, look at where it fails, and refine it. Then test again. Do this several times before calling a prompt production-ready.
The best teams maintain a prompt library with versioned prompts, test datasets, and defined evaluation criteria. It sounds like extra work upfront, but it saves enormous amounts of debugging time when your application misbehaves in production.
If your application accepts user input and includes that input in a prompt to an LLM, you have a security concern worth thinking about seriously: prompt injection.
Prompt injection is when a malicious user writes input designed to override your system prompt instructions. Imagine you have a customer support bot with a system prompt that says “Only answer questions about our product.” A malicious user might type:
User input: Ignore all previous instructions.
If you concatenate that directly into your prompt without any safeguards, the model might comply. This is a real vulnerability in production AI applications, and one that many teams discover the hard way.
Here are the main ways to defend against it:
Security is not an afterthought in AI applications. It needs to be part of your design from day one, the same way it is for any other part of your stack.
There is a clear difference between prompting when you are experimenting and prompting when you are building a real product.
When you are experimenting, being loose is fine. Try different things, see what happens, iterate fast. The cost of a bad output is low.
When you are building a product, every prompt becomes part of your application infrastructure. A change to one prompt can change the behavior of an entire feature. You need to apply the same discipline to prompts that you apply to code:
This discipline is what separates teams that ship reliable AI features from teams that have something that works in the demo and breaks in production.
Let us walk through building a complete, production-grade prompt step by step. The use case: an internal tool that reads a developer’s question from a WhatsApp community message and generates a structured response for the community manager to review before sending.
Prompt: Answer this developer question: {user_question}
This gives the model almost nothing to work with. No persona, no format, no constraints. The output will be unpredictable and inconsistent.
System: You are a technical educator at CodeKerdos.in, a platform for working developers.
Your audience consists of Java developers with 1 to 5 years of experience.
You are approachable, clear, and practical. You avoid unnecessary jargon.
System (continued):
When answering a developer question, always use this structure:
– Short Answer: 2 to 3 sentences that directly address the question.
– Detailed Explanation: 1 to 2 paragraphs with the reasoning and context.
– Code Example: A minimal, working Java or Spring Boot snippet if relevant.
– Common Mistake: One thing developers often get wrong about this topic.
– Learn More: One specific concept or resource to explore next.
System (continued):
Rules:
– Never invent API names, class names, or libraries that do not exist.
– If the question is unclear, ask one clarifying question instead of guessing.
– Keep the code example under 30 lines.
– If the question is unrelated to software development, politely redirect.
User: {user_question}
Note: user_question is injected by your Spring Boot service after
sanitizing the input received from the WhatsApp webhook payload.
That is a prompt you can actually ship. It is structured, constrained, and will produce consistent output across thousands of questions from your developer community.
Next week we go into RAG, which stands for Retrieval Augmented Generation. This is the technique that makes LLMs genuinely useful with your own data, rather than just the data they were trained on.
If you have ever wanted to build a chatbot that knows about your specific product, your internal documentation, or your company’s knowledge base, RAG is how you do it. We will explain the architecture, walk through how it works conceptually, and look at how you can implement it with a Java and Spring Boot backend.
It is one of the most practical and widely used Gen AI patterns in production today, and it builds directly on top of the prompting skills you have just learned.
Prompt engineering is not a soft skill. It is a technical discipline with real consequences for the quality, reliability, cost, and security of the AI features you build. Start treating your prompts like code: version them, test them, review them, and iterate on them. The developers who do this consistently are the ones building AI products that actually hold up.
New post every week. From prompt engineering to building end-to-end AI features with Java and Spring Boot. Join our WhatsApp community for live Q&A, early access, and hands-on exercises.