Microsoft POML : Programming Language for Prompting

Microsoft POML : Programming Language for Prompting

Microsoft Prompt Orchestration Markup Language

I ran into Microsoft’s POML, Prompt Orchestration Markup Language, which is one more thing between me and just writing plain, natural prompts. The kind that feel like actual language.

They say POML gives you “structure, maintainability, and versatility” for prompt engineering. Which is marketing-speak for:

you won’t lose your mind when prompts get long, messy, and reused by 5 different teams across 3 time zones.

My new book on Model Context Protocol is live

Model Context Protocol: Advanced AI Agents for Beginners (Generative AI books)

POML Key Features

POML looks quite strong when it comes to structuring prompt engineering

1. Support for Diverse File Types Beyond Just Images

• Word documents, PDFs, CSVs, audio files, even entire folders can be embedded into the prompt using data components like <document> or <table>.
• You’re not just limited to static text or images anymore. Prompts can now be data-rich and contextual in a way that looks more like a full-blown report than a sentence.

2. Real Coding Experience Inside VS Code

• There’s a VS Code extension specifically for POML, offering:

Live preview

Testing directly inside the editor

Prompt management like writing code, which basically turns prompt engineering into software engineering.

3. Component Categories

• Logical/formatting components: like paragraphs or lists.
• Intention components: like <role>, <task>, <example>.
• Data components: like <image>, <document>, <table>, which inject structured external data.

4. Examples with Real Prompt Engineering Constructs

• Few-shot prompting using <example>, with clearly marked <input> and <output> subcomponents.
• Multimodal prompts: e.g., embedding images with syntax=”multimedia” for LLMs that support vision.
• Alt text fallback: for models that don’t support images, alt text is used instead.

5. Prompt Logic and Control Flow

• Variables using let
• Loops using for, with loop.index access
• Dynamic generation of numbered lists using index + 1

6. QA-Specific Tag

• There’s a <qa> tag to cleanly separate a customer’s question from the response – turning prompts into support agents.

7. Prompt Styling via Stylesheets

• You can attach stylesheets to globally enforce visual or structural rules (like forcing all captions into uppercase).
• It’s not just functional, it’s also about making the prompt readable and maintainable by humans.

8. New Tags Not Mentioned Before

• <hint>: Adds contextual guidance to help the LLM
• <cp> (Captioned Paragraph): Used to create titled sections
• <let> and <for>: Bring templating logic inside prompts

These additions show that POML is far more than just a way to write prompts in a neater format. It’s evolving into a full prompt programming interface, data-aware, reusable, modular, and testable, with tight IDE support. The new details give it the feel of Jupyter Notebooks meets HTML for LLMs

What Is POML Really?

At the core, it’s a markup language. That means it uses tags like HTML does. Tags like: <role>, <task>, and <example>

You use these to break your prompt into pieces. So instead of dumping a 20-line prompt blob into a text file, you now have structure. Think of it like giving your LLM a neatly packed lunchbox instead of a chaotic bag of leftovers.

You can also embed:

<img>    — for images  
<document> — to attach reference text  
<table>  — if your prompt has tabular data

So now your prompt can say, “Hey model, look at this chart” or “Refer to this PDF,” without stuffing the whole thing inline.

It even has a styling system, kind of like CSS. You can tweak how verbose or formal the prompt sounds by applying styles. Think: you change tone without rewriting the content.

And yeah, templating. That’s the part where things get really close to programming:

• You can set variables using {{ variable_name }}
• Add <let> to define them
• Use conditionals like if statements
• Even loops.

Which means your prompt can say: “If the user’s language is French, do this. Otherwise, do that.” That kind of stuff.

The Microsoft Touch: Dev Tools

This isn’t just a spec. Microsoft shipped a whole kit.

• VSCode extension: autocomplete, tag suggestions, error highlighting, previews.
• Python & Node.js SDKs: lets you plug POML prompts into your app logic.
• No C#/.NET SDK though, which is wild. Microsoft made this thing and didn’t even toss a bone to its own main dev ecosystem. Classic.

You write a prompt in POML, the SDK compiles it into a plain-text prompt, then that goes to the LLM. Think of POML as a structured blueprint that gets flattened right before the model sees it.

Example

<poml>
  <role>You are a witty science communicator who uses analogies and sarcasm to make complex ideas fun.</role>
  <task>Describe how a rocket launches into space, using the image provided for reference.</task>

  <img src="rocket_launch_diagram.png" alt="Diagram of rocket launch" />

  <output-format>
    Keep it punchy, humorous, and under 120 words.
    Start with "Alright, buckle up, space cadet!".
  </output-format>
</poml>

This example sets a playful, sarcastic tone for the LLM, defines a task related to rocket launches, and includes an image for visual grounding. The output format guides the model to be punchy and humorous, with a specific opening line and word limit. Using POML, this structured prompt can be clearly organized, reused, and tested with a vision-enabled LLM for consistent behavior.

What Folks Online Are Saying

I dug around Reddit and Hacker News. Here’s what’s echoing:

• “LLMs don’t care about formatting unless they were trained to.”
  Fair. If your model doesn’t know how to read tags, it might just ignore them or worse choke on them.
• “This feels like programming LLMs to program.”
  Yup. The tool that writes prompts is now a program, which builds programs, which generate code.
• “This isn’t structure for the LLM, it’s structure for devs who are writing prompts in messy workflows.” Probably the most honest take. This helps humans stay sane, especially when prompts aren’t just one-offs.
• And yeah, the C# SDK absence? It’s not just me. Other folks raised eyebrows too.

What I Actually Like About It

Okay, now that I’ve dunked on it a bit… here’s what makes me pause before throwing it out:

• My current prompts are just… blobs. Changing one word sometimes screws up the whole output. Adding structure could help me see what’s changing and why.
• Repeating patterns, like the same structure across different tasks, might make things more predictable for the model, especially if it was trained on similar markup.
• The dev tooling is smooth. VSCode autocomplete is the kind of thing that saves real time, especially when the syntax gets clunky.
• Writing reusable prompts with variables and logic? That could cut down a lot of copy-paste junk.

Some issues

• It’s one more abstraction. I already had text → LLM. Now it’s text → markup → compile → LLM. More moving parts. More failure points.
• It feels heavy. Like we’re building 2005-era websites again. Verbose tags. Bracket soup. It’s not elegant.
• No .NET support. I write tools in C#. This means I’d have to Frankenstein together something just to use it in my stack.
• And honestly, unless the model was trained to expect this format… what’s the point? You’re just wrapping clean text with noisy tags.

What I’d Tell You If You’re New To This

Try it. But don’t commit.

Take two prompts you already wrote. Rewrite them using POML. Use the VSCode extension. See how it feels. Don’t read the whole spec, just mimic some examples.

Ask yourself:

• Is it easier to maintain?
• Did it help you think more clearly about what the model’s supposed to do?
• Did it make collaboration easier with other folks?

And crucially: Did it break anything?

If the LLM’s behavior improved, or at least didn’t get worse, maybe it’s worth using. If it made everything slower, more complex, and less readable, dump it. Go back to plain text. There’s no prize for using the latest format.

And yeah, if you ever want to write a weird, templated prompt full of <let> loops or <img> embeds, I’ll be around. This stuff is messy, but sometimes that’s where the good ideas hide.

Explore more here

GitHub – microsoft/poml: Prompt Orchestration Markup Language

---

Microsoft POML : Programming Language for Prompting was originally published in Data Science in Your Pocket on Medium, where people are continuing the conversation by highlighting and responding to this story.