Factory AI Complete Guide: Setup, Droids, and Real-World Workflows

After testing every AI coding tool available - Copilot, Cursor, Codeium, and more - I can definitively say Factory AI is different. It's not another line-completion tool. It's a platform with specialized AI agents (Droids) that handle your entire development workflow: coding, debugging, research, documentation, and production incidents.

This is the complete Factory AI guide based on real usage. You'll learn setup, how each Droid works, context management, integrations, and workflows for students, professional developers, and teams.

What is Factory AI?

Factory AI is a development platform with specialized AI agents (called Droids) that handle different parts of software development. While GitHub Copilot suggests code line-by-line, Factory understands your entire project, executes commands, and maintains context across sessions.

Key difference: Factory integrates with your entire development workflow - Jira tickets, Notion docs, Slack discussions, Sentry errors, and your codebase. It uses all this context to make better decisions.

Meet the Specialized Droids

Each Droid is optimized for specific tasks with specialized tools and system prompts:

Complete Setup Guide

Factory works through CLI, Web App, and IDE. I recommend starting with CLI as it's the most powerful.

1Install the CLI

Open your terminal and run one command:

On first launch, authenticate through your browser. You'll get free tokens immediately.

2Set Up AGENTS.md (Critical)

Create an AGENTS.md file in your project root. This is how Factory understands your project:

This file gets loaded at conversation start. Factory uses it to understand your conventions, testing approach, and project structure.

3First Session: Understand Your Codebase

Start by asking Factory to analyze your project:

Factory will read your code and respond with insights about:

• Project architecture and patterns
• Test frameworks used
• Code conventions
• How components connect

Spec Mode: Plan Before Building

Factory's killer feature is Specification Mode. Press Shift-Tab to cycle through modes:

• Default: Ask permission for every action
• Automatic: Make reversible changes automatically
• Planning (Spec Mode): Generate detailed spec before coding

How Spec Mode Works

Switch to Spec mode and describe what you want:

Factory generates a complete specification:

• Acceptance Criteria: What "done" looks like
• Implementation Plan: Step-by-step approach
• Technical Details: Libraries, patterns, security
• File Changes: Which files affected
• Testing Strategy: What tests needed

Three Autonomy Levels

After approving a spec, choose how much control to give Factory:

1. Low (Manual approval): Approve every file edit
2. Medium (Safe commands): Reversible changes automatic, risky ones need approval
3. High (Full autonomy): Factory handles everything

Context Management: How Factory Sees Your Project

Factory's intelligence comes from layered context management:

Layer 1: AGENTS.md

Repository-level conventions loaded at every session start. You can have multiple:

• `/AGENTS.md` - Root conventions
• `/packages/api/AGENTS.md` - API-specific rules
• `/packages/web/AGENTS.md` - Frontend patterns

Layer 2: Dynamic Code Context

When you ask a question, Factory automatically:

• Searches for relevant files
• Picks out important code sections
• Avoids duplicating context
• Lazy loads additional context as needed

You can also @mention specific files, but letting Factory figure it out usually works better.

Layer 3: Tool Integrations

Connect Factory to where your context lives:

Layer 4: Memory

Factory maintains two types of persistent memory:

User Memory (Private to you):

• Your dev environment setup
• Your work history
• Your preferences and patterns

Organization Memory (Shared):

• Company-wide conventions
• Security requirements
• Architecture patterns

Connecting Your Dev Tools

Sign in to Factory web app and connect integrations:

Essential: Source Control

Connect GitHub or GitLab (cloud or self-hosted). Required for Droids to read and write code.

High Impact Integrations

Notion/Google Docs: Architecture decisions, design docs, onboarding guides

Jira/Linear: Ticket descriptions, acceptance criteria, dependencies

Slack: Technical discussions, decisions made in channels

Sentry/Datadog: Error traces, performance metrics, incident history

Advanced Features

Droid Exec: Headless Automation

Run Factory non-interactively for CI/CD, cron jobs, or pre-commit hooks:

Factory executes everything and exits. Perfect for automation.

Custom Droids

Create specialized sub-agents in `.factory/droids/` as markdown files:

Custom Slash Commands

Automate routine actions. Define in `.factory/commands/`:

Bring Your Own Model Key

Use Factory's orchestration with your own API key. Access via /model command.

Real-World Workflows

Here's how different users leverage Factory's capabilities:

For Students: Learning + Building

Debugging Assignments:

Factory analyzes, identifies the issue (likely O(n²) complexity), and teaches you why it breaks on large datasets.

Course Projects That Impress:

Factory generates tech stack, database schema, API design, and implementation roadmap. Most students submit basic apps - you can submit production-quality tools.

Learning New Frameworks:

Factory teaches concepts while building practical examples you can learn from.

For Professional Developers: Features + Debugging

Feature Development with Spec Mode:

Factory reads your existing auth code, generates a complete spec matching your patterns, then implements with tests.

Production Debugging (Reliability Droid):

Reliability Droid fetches Sentry traces, analyzes logs, identifies connection pool exhaustion, and suggests fixes with monitoring.

Refactoring Legacy Code:

Factory analyzes dependencies, proposes module structure, refactors with tests to ensure nothing breaks.

For Teams: Collaboration + Context

Implementing from Tickets:

Factory fetches ticket details, reads Notion architecture doc, checks team conventions in Org Memory, and implements following all context.

Onboarding New Team Members:

Knowledge Droid searches codebase, reads documentation, and provides comprehensive overview with file references.

Incident Response:

Reliability Droid analyzes metrics, identifies slow database query, suggests index addition, documents resolution in incident report.

Pricing & Tokens

Factory uses token-based pricing, not subscriptions:

• Free Tier: Generous starting tokens (use promo for 40M)
• Pay-as-you-go: Buy tokens as needed
• Enterprise: Unlimited for teams

Factory AI vs Alternatives: Complete Comparison

Based on 3 months testing every major AI coding tool:

When to Use Each Tool

Use Factory AI when:

• Building complete features (not just editing code)
• Working on complex projects with lots of context
• Need production debugging and incident response
• Want integrations with your entire dev stack
• Planning before implementation (Spec Mode)

Use Copilot when:

• You want simple autocomplete while typing
• Working in IDE with minimal context needed
• Your organization already pays for GitHub Enterprise

Use Cursor when:

• You want a purpose-built AI IDE
• Interactive code editing is your main workflow
• Willing to switch from VS Code

Use Codeium when:

• You want free forever autocomplete
• Basic suggestions are sufficient
• Budget is zero

Common Questions

Is Factory free for students?

Yes, you get free tokens to start (use promo link for 40M). This covers weeks of regular usage. Additional tokens are affordable pay-as-you-go.

Does it work offline?

No, Factory requires internet as it uses cloud-based AI models.

Which languages are supported?

All major languages: Python, JavaScript, TypeScript, Java, C++, Go, Rust, Ruby, PHP, and more.

Can I use this for homework?

Check your school's policy. Use Factory to learn and understand, not to skip learning. Most universities allow AI assistance similar to Stack Overflow or tutoring.

How is my code used?

Factory uses your code only to provide assistance. It's not used for training models. See privacy policy.

What if I'm not happy with generated code?

Hit Escape to reject and provide feedback. Factory learns from corrections. With proper AGENTS.md setup and clear prompts, accuracy is very high.

Video Tutorial

Official Resources

• Factory AI Docs: docs.factory.ai
• CLI Quickstart: Getting Started Guide
• Factory Website: factory.ai
• Context Guide: Managing Context

Key Takeaways

After months of real-world usage, here's what makes Factory AI worth using:

What Factory does best:

• Context understanding: AGENTS.md + Memory + Integrations = AI that actually understands your project
• Specialized agents: Different Droids for different tasks (not one-size-fits-all)
• Planning mode: Spec Mode prevents costly mistakes by planning before coding
• Team workflows: Org Memory ensures everyone's AI follows same conventions
• Production debugging: Reliability Droid handles incidents like an on-call engineer

Success formula:

1. Create detailed AGENTS.md (this is critical)
2. Connect your integrations (Jira, Notion, Slack, etc.)
3. Start with low autonomy, build trust
4. Use Spec Mode for complex features
5. Let Factory handle boilerplate while you focus on architecture

Reality check: Factory won't magically make you a better developer. But it will handle repetitive tasks, help you understand unfamiliar codebases faster, and catch issues before they hit production. The developers who succeed with Factory use it to amplify their skills, not replace them.

Start with the free tokens, experiment with different Droids, and integrate with your workflow. The learning curve is real, but once you understand how to manage context effectively, Factory becomes indispensable.