ZenML LLMOps Roadmap Feedback

Welcome! As one of our 5,000 newsletter subscribers, you're part of the community that's been with us since the beginning.

Now we need your help again—this time to bring the same MLOps standardization to the chaos of LLM development.

Your 5 minutes here will shape the tools we all use tomorrow.

What's your current role?

Which challenges are you currently facing with LLM/Agent development?

How mature is your organization's LLM/Agent deployment?

What best describes the complexity of your LLM/Agent use cases?

We're trying to understand the technical depth of what you're building

Rank these proposed ZenML LLM components by importance to you

Drag and drop to rank from 1 (most important) to 4 (least important) based on which of these (proposed) ZenML Stack Components would provide the most value for your workflows.

Use the up and down arrow keys to change the rank.

1

LLM Provider Component - Unified API management, automatic cost tracking, and provider fallbacks (e.g., LiteLLM, Portkey)

1

Prompt Manager - Version control and templating for prompts across your pipelines

1

LLM Evaluator - Standardized interface, LLM-as-judge, visualizations in the ZenML dashboard (e.g. OpenAI evals, RAGAS, deepeval)

1

LLM Experiment Tracker - Full observability and session tracing for LLM calls (e.g., LangSmith, Langfuse, Braintrust)

What type of LLM/Agent applications are you building?

How do you prefer to deploy LLM agents in production?

Think about how agents fit into your overall system architecture

Do you currently combine traditional ML with LLMs in your pipelines?

Why do you keep traditional ML and LLM systems separate? (Optional)

This helps us understand integration challenges

ZenML LLMOps Roadmap Feedback

What's your current role?

Which challenges are you currently facing with LLM/Agent development?

How mature is your organization's LLM/Agent deployment?

What best describes the complexity of your LLM/Agent use cases?

Rank these proposed ZenML LLM components by importance to you

What type of LLM/Agent applications are you building?

How do you prefer to deploy LLM agents in production?

Do you currently combine traditional ML with LLMs in your pipelines?

Why do you keep traditional ML and LLM systems separate? (Optional)

Which tools are you currently using for LLMOps?

What's your biggest frustration with current LLMOps tooling?

If ZenML offered the LLMOps features described above, how likely would you be to adopt them?

What timeline are you working with for improving your LLMOps practices?

What's one LLMOps capability you wish existed that no tool currently provides?

Would you be interested in:

Size of the team(s) working on your LLMOps projects and features:

If you'd be ok with us following up with you, feel free to submit your email below

Anything else you'd like to tell us?