This directory contains comprehensive tests for the Swarms framework, covering all major components including agents, workflows, tools, utilities, and more.

• test_comprehensive_test.py - Main comprehensive test suite that runs all major Swarms components
• test___init__.py - Package initialization tests
• requirements.txt - Test dependencies (swarms, pytest, matplotlib, loguru)

Tests for individual agent functionality and behavior:

/agents/ - Core agent functionality

• test_agent_logging.py - Agent logging and monitoring capabilities
• test_create_agents_from_yaml.py - YAML-based agent creation
• test_litellm_args_kwargs.py - LiteLLM argument handling
• test_llm_args.py - LLM argument processing
• test_llm_handling_args.py - LLM argument management
• test_tool_agent.py - Tool-enabled agent functionality

/benchmark_agent/ - Agent performance and benchmarking

• test_agent_benchmark_init.py - Agent benchmark initialization
• test_agent_exec_benchmark.py - Agent execution benchmarking
• test_auto_test_eval.py - Automated test evaluation
• test_github_summarizer_agent.py - GitHub summarization agent
• test_profiling_agent.py - Agent performance profiling

Tests for Swarms structural components and workflows:

• test_agent.py - Core Agent class functionality
• test_agent_features.py - Agent feature testing
• test_agent_rearrange.py - Agent rearrangement capabilities
• test_agentrearrange.py - Alternative agent rearrangement tests
• test_airflow_swarm.py - Airflow integration
• test_auto_swarm_builder_fix.py - Auto swarm builder fixes
• test_auto_swarms_builder.py - Automated swarm construction
• test_base_workflow.py - Base workflow functionality
• test_base.py - Base class implementations
• test_board_of_directors_swarm.py - Board of directors swarm pattern
• test_concurrent_workflow.py - Concurrent workflow execution
• test_conversation.py - Conversation management
• test_forest_swarm.py - Forest swarm architecture
• test_graph_workflow_comprehensive.py - Graph-based workflows
• test_groupchat.py - Group chat functionality
• test_majority_voting.py - Majority voting mechanisms
• test_moa.py - Mixture of Agents (MoA) testing
• test_multi_agent_collab.py - Multi-agent collaboration
• test_multi_agent_orchestrator.py - Multi-agent orchestration
• test_reasoning_agent_router_all.py - Reasoning agent routing
• test_recursive_workflow.py - Recursive workflow patterns
• test_round_robin_swarm.py - Round-robin swarm scheduling
• test_sequential_workflow.py - Sequential workflow execution
• test_spreadsheet.py - Spreadsheet swarm functionality
• test_swarm_architectures.py - Various swarm architectures
• test_yaml_model.py - YAML model configuration

Tests for tool integration and functionality:

• test_base_tool.py - Base tool class functionality
• test_output_str_fix.py - Output string formatting fixes
• test_parse_tools.py - Tool parsing and execution
• test_support_mcp.py - MCP (Model Context Protocol) support

Tests for utility functions and helpers:

• test_acompletions.py - Async completion handling
• test_auto_check_download.py - Automatic download checking
• test_display_markdown_message.py - Markdown message display
• test_docstring_parser.py - Docstring parsing utilities
• test_extract_code_from_markdown.py - Code extraction from markdown
• test_formatter.py - Text formatting utilities
• test_litellm_wrapper.py - LiteLLM wrapper functionality
• test_math_eval.py - Mathematical expression evaluation
• test_md_output.py - Markdown output handling
• test_metrics_decorator.py - Metrics collection decorators
• test_pdf_to_text.py - PDF to text conversion
• test_try_except_wrapper.py - Error handling wrappers

Tests for artifact management and versioning:

• test_artifact_main.py - Core artifact functionality
• test_artifact_output_types.py - Artifact output type handling

Tests for communication and conversation management:

• test_conversation.py - Conversation handling and persistence

Advanced testing with benchmarking and performance analysis:

• aop_benchmark.py - Comprehensive AOP benchmarking suite
• test_data/ - Benchmark data and results
  • aop_benchmark_data/ - Benchmark results and visualizations
  • image1.jpg, image2.png - Test images

Tests for telemetry and monitoring:

• test_user_utils.py - User utility telemetry

Install test dependencies:

pip install -r requirements.txt

pytest

# Run agent tests
pytest agent/

# Run structure tests
pytest structs/

# Run utility tests
pytest utils/

# Run tool tests
pytest tools/

# Run comprehensive test suite
pytest test_comprehensive_test.py

# Run specific test file
pytest structs/test_agent.py

pytest --cov=swarms --cov-report=html

• Agent Functionality: Complete testing of agent creation, execution, and management
• Workflow Testing: Various workflow patterns including sequential, concurrent, and recursive
• Tool Integration: Testing of tool parsing, execution, and MCP support
• Performance Benchmarking: AOP benchmarking with multiple LLM providers
• Error Handling: Comprehensive error handling and recovery testing

• Benchmark results with CSV and Excel exports
• Performance visualizations (PNG charts)
• Test images for multimodal testing
• Conversation cache files for persistence testing

The AOP benchmark tests support multiple LLM providers:

• OpenAI (GPT-4o, GPT-4o-mini, GPT-4-turbo)
• Anthropic (Claude 3.5 Sonnet, Claude 3 Haiku, Claude 3 Sonnet)
• Google (Gemini 1.5 Pro, Gemini 1.5 Flash)
• Meta (Llama 3.1 8B, Llama 3.1 70B)

Tests require the following environment variables:

• OPENAI_API_KEY - OpenAI API key for testing
• Additional API keys for other providers (optional)

• Maximum agents: 20 (configurable in AOP benchmark)
• Requests per test: 20
• Concurrent requests: 5
• Timeout settings: Configurable per test type

The AOP benchmark suite provides:

• Performance metrics across multiple LLM providers
• Memory usage tracking
• Response time analysis
• Throughput measurements
• Visual performance reports

pytest -v

pytest --pdb

Tests use Loguru for comprehensive logging. Check console output for detailed test execution logs.

When adding new tests:

1. Follow the existing directory structure
2. Use descriptive test names
3. Include proper docstrings
4. Add appropriate fixtures and mocks
5. Update this README if adding new test categories

The test suite aims for comprehensive coverage of:

• ✅ Agent creation and execution
• ✅ Workflow patterns and orchestration
• ✅ Tool integration and execution
• ✅ Utility functions and helpers
• ✅ Error handling and edge cases
• ✅ Performance and benchmarking
• ✅ Communication and conversation management
• ✅ Artifact management and versioning