LangChain for Developers

Build AI Apps Without Reinventing Everything From Scratch

Arjun had been working on the same AI feature for three weeks.

It started simple enough. His team wanted a chatbot that could answer questions about their product documentation. He built the RAG pipeline from scratch: document loader, text splitter, embedding calls, vector store, retrieval logic, prompt assembly, LLM call, response parsing. Each piece took longer than expected. Each piece had its own edge cases. And when he finally got it working, his manager walked over and said: “Great. Now can we also add a feature where it searches the web if it cannot find the answer in the docs?”

Arjun stared at his screen. Adding web search meant a whole new retrieval path. A new tool. New decision logic to figure out when to use docs versus web. New prompt templates. New response formatting. He had spent three weeks building what was essentially plumbing, and now the plumbing needed to change again.

A colleague in the next row heard the conversation and turned around. “Are you using LangChain?” she asked.

Arjun had heard the name. He had seen it in every Gen AI tutorial. He had assumed it was just another library for making LLM API calls, something he could do himself with a few lines of Python. He had not thought it was worth learning.

“I thought I was saving time by not learning a framework. I was actually spending three weeks building a worse version of one.”

That is the trap a lot of developers fall into with LangChain. It looks like abstraction for abstraction’s sake until you hit the exact problems it was built to solve. Then it suddenly looks like the most obvious tool in the world.

This week, we cover what LangChain actually is, what it gives you that raw API calls do not, where it fits into a production AI application, and where you should not use it. By the end, you will know exactly when to reach for it and when to keep things simple.

What LangChain actually is

LangChain is an open-source framework for building applications powered by large language models. The core idea is simple: most AI applications are not a single LLM call. They are a sequence of steps, retrievals, decisions, tool invocations, and LLM calls chained together to accomplish something meaningful.

LangChain gives you standardized building blocks for each of those steps, plus the connective tissue to link them together. Instead of writing custom code for every integration, you compose your application from components that already handle the boring parts: retries, output parsing, prompt templating, memory management, tool calling.

It started as a Python library. There is now a JavaScript/TypeScript version called LangChain.js. For Java developers, Spring AI covers much of the same ground using Spring-native patterns, which we will come back to later in this post.

What LangChain is not

It is not a model. It does not run inference. It does not host anything. It is a framework that sits between your application code and the LLM APIs you are already calling. Think of it the way you think of Spring Boot: it does not replace Java; it gives Java developers a structured way to build applications without reinventing common patterns.

The core building blocks

LangChain has a lot of components. Learning all of them at once is overwhelming and unnecessary. Here are the ones you will use in almost every real application.

1. Models

LangChain wraps LLM providers behind a consistent interface. Whether you are calling OpenAI, Anthropic, Groq, Gemini, or a local Ollama model, the API your code uses looks the same. This means switching providers is a one-line change, not a refactor.

# Swap providers without changing your application logic
from langchain_openai import ChatOpenAI
from langchain_anthropic import ChatAnthropic
from langchain_groq import ChatGroq
 
# These all behave identically from your code's perspective
llm = ChatOpenAI(model="gpt-4o")
llm = ChatAnthropic(model="claude-sonnet-4-6")
llm = ChatGroq(model="llama-3.1-70b-versatile")
 
# The call is always the same
response = llm.invoke("Explain RAG in one paragraph")

In a real project, this matters more than it sounds. You might prototype with OpenAI, switch to Groq for speed, and fall back to a local Ollama model for privacy-sensitive workloads, all without rewriting your application.

2. Prompt Templates

Prompt templates are reusable, parameterized prompts. Instead of building prompt strings with f-strings scattered across your codebase, you define them once, version them, and inject variables at runtime.

from langchain_core.prompts import ChatPromptTemplate
 
template = ChatPromptTemplate.from_messages([
    ("system", """You are a helpful assistant for {company_name}.
                  Answer only using the context provided.
                  If the answer is not in the context, say you do not know."""),
    ("human", "Context: {context}\n\nQuestion: {question}")
])
 
# At runtime, inject variables
prompt = template.invoke({
    "company_name": "CodeKerdos",
    "context": retrieved_chunks,
    "question": user_question
})

This is a small thing that pays off enormously at scale. When you have 20 different prompt templates across a complex application, having them as named, versioned objects instead of inline strings is the difference between a maintainable codebase and a debugging nightmare.

3. Output Parsers

LLMs return text. Your application usually needs structured data. Output parsers bridge that gap. You define the structure you want and LangChain handles the parsing and validation.

from langchain_core.output_parsers import JsonOutputParser
from pydantic import BaseModel
 
class TicketClassification(BaseModel):
    issue_type: str
    urgency: str   # low, medium, high
    product_area: str
    needs_escalation: bool
 
parser = JsonOutputParser(pydantic_object=TicketClassification)
 
# Chain: prompt -> llm -> parser
chain = prompt_template | llm | parser
 
# Returns a validated TicketClassification object, not raw text
result = chain.invoke({"ticket": raw_ticket_text})
print(result.urgency)   # "high"
print(result.needs_escalation)  # True

The pipe operator between components is LangChain’s way of expressing chains. Each component takes input, processes it, and passes the output to the next. This is the LCEL syntax, which stands for LangChain Expression Language, and it is how most modern LangChain code is written.

4. Memory

Memory is what turns a single-turn Q&A system into a real conversation. Without memory, every message to the LLM is stateless: the model has no idea what was said two messages ago. With memory, previous exchanges are included in the context automatically.

from langchain_community.chat_message_histories import ChatMessageHistory
from langchain_core.runnables.history import RunnableWithMessageHistory
 
# Store per-session chat history
store = {}
 
def get_session_history(session_id: str) -> ChatMessageHistory:
    if session_id not in store:
        store[session_id] = ChatMessageHistory()
    return store[session_id]
 
# Wrap your chain with history management
chain_with_memory = RunnableWithMessageHistory(
    chain,
    get_session_history,
    input_messages_key="question",
    history_messages_key="chat_history"
)
 
# Same session_id = model remembers previous turns
chain_with_memory.invoke(
    {"question": "What is RAG?"},
    config={"configurable": {"session_id": "user_123"}}
)

In production you would replace the in-memory store with Redis or a database, but the interface stays the same. LangChain handles injecting the history into each prompt automatically.

5. Document Loaders and Text Splitters

LangChain ships with loaders for almost every document format you will encounter: PDF, Word, HTML, CSV, JSON, Notion pages, Confluence pages, Google Docs, S3 files, web URLs, and many more. Combined with text splitters that handle chunking intelligently, this takes what was an hour of custom parsing code and reduces it to a few lines.

from langchain_community.document_loaders import PyPDFLoader, WebBaseLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
 
# Load a PDF
loader = PyPDFLoader("company_policy.pdf")
pages = loader.load()
 
# Or load a webpage
web_loader = WebBaseLoader("https://codekerdos.in/docs")
web_pages = web_loader.load()
 
# Split into chunks with overlap
splitter = RecursiveCharacterTextSplitter(
    chunk_size=500,
    chunk_overlap=50,
    separators=["\n\n", "\n", ". ", " "]
)
chunks = splitter.split_documents(pages)

Building a RAG chain with LangChain

Let us put the building blocks together and build a complete RAG chain. This is the kind of thing Arjun spent three weeks building from scratch. With LangChain it looks like this:

from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from langchain_community.vectorstores import Chroma
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnablePassthrough
from langchain_core.output_parsers import StrOutputParser
 
# 1. Set up the vector store (already indexed)
vectorstore = Chroma(
    persist_directory="./chroma_db",
    embedding_function=OpenAIEmbeddings()
)
retriever = vectorstore.as_retriever(search_kwargs={"k": 4})
 
# 2. Define the prompt template
prompt = ChatPromptTemplate.from_template("""
Answer the question using only the context below.
If the answer is not in the context, say you do not know.
 
Context: {context}
 
Question: {question}
""")
 
# 3. Build the chain
llm = ChatOpenAI(model="gpt-4o-mini")
 
rag_chain = (
    {"context": retriever, "question": RunnablePassthrough()}
    | prompt
    | llm
    | StrOutputParser()
)
 
# 4. Invoke
answer = rag_chain.invoke("What is the refund policy for enterprise customers?")
print(answer)

That is a fully functional RAG chain in under 30 lines. The retriever automatically embeds the query, searches the vector store, and passes the results as context. The chain handles the prompt assembly and output parsing. You own the business logic. LangChain owns the plumbing.

Now when your manager asks you to add web search as a fallback, you add a second retriever and a router. You do not rewrite the entire pipeline.

Agents: when the model needs to make decisions

Everything we have covered so far is a chain: a fixed sequence of steps that runs the same way every time. Chains are powerful but rigid. They do not decide what to do based on the situation.

Agents are different. An agent uses the LLM itself to decide which action to take next, based on the user’s input and the results of previous actions. The model becomes the orchestrator, not just the responder.

Here is a concrete example. Imagine a developer assistant that can:

  • Search your internal documentation
  • Search the web for recent information
  • Execute Python code to do calculations
  • Look up a Jira ticket by ID

A chain cannot handle this because you do not know upfront which tools the user’s question will require. An agent can, because it reasons through the problem: “The user asked about a Jira ticket number, so I should call the Jira tool. The result mentions a deployment, so I should also search the docs for deployment procedures.”

How agents work internally

The most common agent pattern in LangChain is called ReAct, which stands for Reasoning and Acting. The model is given a set of available tools and follows a loop:

  1. Thought: the model reasons about what it needs to do next.
  2. Action: it selects a tool and calls it with specific inputs.
  3. Observation: it receives the tool’s output.
  4. Repeat: it reasons again based on the observation, until it has enough information to give a final answer.
from langchain.agents import create_react_agent, AgentExecutor
from langchain_community.tools import DuckDuckGoSearchRun
from langchain_core.tools import tool
 
# Define a custom tool
@tool
def get_jira_ticket(ticket_id: str) -> str:
    """Fetches details of a Jira ticket by its ID."""
    # Your actual Jira API call here
    return jira_client.get_issue(ticket_id)
 
# Define the agent with tools
tools = [DuckDuckGoSearchRun(), get_jira_ticket]
 
agent = create_react_agent(llm, tools, prompt)
agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)
 
# The agent decides which tools to use and in what order
result = agent_executor.invoke({
    "input": "Look up PROJ-4821 and tell me if the fix has been deployed"
})

With verbose=True you can watch the agent’s reasoning in real time: which tool it chose, what it passed in, what it got back, and how it decided to proceed. This is enormously helpful for debugging agent behavior.

When to use agents vs chains

This is one of the most common questions developers ask when they first learn LangChain. The answer is straightforward:

  • Use a chain when the sequence of steps is fixed and known upfront. RAG pipelines, document summarization, classification tasks. Chains are faster, cheaper, more predictable, and easier to debug.
  • Use an agent when the steps depend on the user's input and cannot be determined upfront. Multi-tool assistants, open-ended research tasks, anything where the model needs to make decisions about what to do next.

Agents are slower and more expensive per request because they involve multiple LLM calls in the reasoning loop. They also fail in less predictable ways. Do not default to agents because they sound more impressive. Use chains wherever you can and reach for agents only when the task genuinely requires dynamic decision-making.

LangSmith: the observability layer you need

One of the hardest parts of building with LLMs is debugging. When your chain returns a wrong answer, where do you look? Which step failed? What did the prompt actually look like when it was sent? What did the retriever return? How long did each step take?

LangSmith is the observability platform built specifically for LangChain applications. It traces every run of your chain or agent, showing you the exact input and output of every step, the latency of each component, token counts, costs, and errors. It is the difference between debugging by guessing and debugging with data.

# Enable LangSmith tracing with two environment variables
import os
os.environ["LANGCHAIN_TRACING_V2"] = "true"
os.environ["LANGCHAIN_API_KEY"] = "your_langsmith_api_key"
 
# Every chain and agent run is now automatically traced
# No other code changes needed

LangSmith has a generous free tier. Turn it on from day one of any LangChain project. It will save you hours of debugging time, especially once you start building agents whose multi-step reasoning is impossible to follow from plain console logs.

  • ChatClient for wrapping LLM providers behind a consistent interface
  • PromptTemplate for reusable, parameterized prompts
  • VectorStore abstraction for Pinecone, pgvector, Chroma, and others
  • RAG advisors that handle retrieval and context injection automatically
  • Tool/function calling support for building agent-like behavior

The philosophy is the same as LangChain, applied to the Spring ecosystem. You get vendor independence, structured composition, and Spring-native patterns like dependency injection and auto-configuration.

That said, learning LangChain even as a Java developer is worthwhile for two reasons. First, almost all the Gen AI literature, tutorials, and open-source projects use LangChain examples. Being able to read and understand them makes you a more effective engineer. Second, many real-world AI teams use Python for the AI layer and Java for the rest of the stack. Knowing both makes you genuinely cross-functional.

When NOT to use LangChain

LangChain is a powerful framework but it is not always the right choice. Here is when you should skip it:

Your use case is a single LLM call

If your application sends one prompt and processes one response, you do not need a framework. The OpenAI SDK or Anthropic SDK directly is simpler, faster, and has fewer moving parts. LangChain adds value when you have complexity to manage. Do not introduce complexity just to use it.

You need maximum performance control

LangChain’s abstractions add overhead. For applications where every millisecond matters and you have already profiled and optimized everything else, the framework layer might be something you want to bypass in favor of direct API calls with custom connection pooling and caching.

Your team finds the abstractions confusing

LangChain has a learning curve, and its API surface has changed significantly across versions. If your team is spending more time fighting the framework than building features, it is legitimate to step back and write the orchestration logic yourself. A well-structured custom implementation is better than a poorly understood framework dependency.

You are in a Java-only shop

If your team runs entirely on Java and has no Python capability, Spring AI is a better fit than forcing LangChain into the stack. Do not introduce a new language just to use a framework. The patterns matter more than which tool implements them.

A practical starting point

If you want to get hands-on with LangChain this week, here is the fastest path to a working application:

  1. Install LangChain and pick an LLM: pip install langchain langchain-openai. Set your OPENAI_API_KEY. Run your first chain with a simple prompt template.
  2. Build a basic RAG chain: take 5 documents you care about, load them with PyPDFLoader, split them, store in Chroma, and wire up a retrieval chain. Ask it questions. See where it fails.
  3. Add memory: wrap your chain with RunnableWithMessageHistory. Have a multi-turn conversation with your RAG bot. Watch how adding memory changes the quality of follow-up answers.
  4. Turn on LangSmith: add the two environment variables. Run your chain a few times and explore the traces. You will immediately see what is happening inside the black box.
  5. Try a simple agent: add DuckDuckGo search as a tool and build a ReAct agent. Ask it something that requires both your docs and a web search. Watch it reason through the problem.

Each of those steps builds directly on the last. By the end of that sequence you will have hands-on experience with every major LangChain concept, and a working application to show for it.

What is coming in Week 6?

Next week we go end to end. We build a complete RAG chatbot using Spring Boot, Spring AI, and pgvector, from scratch, with real code for every layer: document ingestion, vector storage, retrieval, prompt assembly, streaming responses, and a simple frontend to tie it all together.

If you have been reading this series to understand the theory, Week 6 is where everything becomes concrete. We will cover every decision point along the way and explain why each piece is built the way it is. It is the most practical post in the series and the one most directly applicable to the kind of work you can put on your resume tomorrow.

Key Takeaway

LangChain is not magic, and it is not overkill. It is a framework that solves real problems that every developer building AI applications will eventually hit: vendor lock-in, prompt management, output parsing, memory, tool orchestration, and observability. Learn the concepts. Use it where it fits. Skip it where it does not. The goal is always to ship something that works, not to use the most impressive tooling.

Follow the full series at codekerdos.in

New post every week. Practical Gen-AI content built for Java and Spring Boot developers. Join our WhatsApp community for early access, live Q&A, and hands-on exercises.

Scroll to Top