Huginn - OSINT Data Collection Engine

Automated OSINT collection engine for pr-cybr's rav3n-n3t framework — gathers and structures intelligence feeds for analysis by muninn.

Huginn is an OSINT (Open Source Intelligence) data collection tool designed to gather information from various public sources including RSS feeds, APIs, and other structured data sources. The collected data is structured and saved in JSON format for downstream analysis by Muninn.

• Multi-Source Collection: Fetch data from RSS feeds, APIs, and other OSINT sources
• Structured Output: Export collected data in JSON format for easy processing
• Configurable Sources: Define data sources via YAML configuration
• Extensible Architecture: Easy to add new data source types

huginn/
├── src/
│   └── huginn.py          # Main collection script
├── config/
│   ├── sources.yaml       # Data source configuration
│   └── sources.example.yaml  # Example configuration
├── output/                 # Collected data output directory
├── requirements.txt        # Python dependencies
└── README.md              # This file

1. Clone the repository:

git clone https://github.com/PR-CYBR/huginn.git
cd huginn

2. Install dependencies:

pip install -r requirements.txt

3. Configure your data sources:

cp config/sources.example.yaml config/sources.yaml
# Edit config/sources.yaml with your sources

Data sources are configured in config/sources.yaml. Example configuration:

sources:
  - name: "Security News Feed"
    type: "rss"
    url: "https://example.com/security/feed.xml"
    enabled: true
  
  - name: "Threat Intelligence API"
    type: "api"
    url: "https://api.example.com/threats"
    enabled: true

Run the data collection:

python src/huginn.py

The collected data will be saved to the output/ directory in JSON format with timestamps.

Collected data is saved as JSON with the following structure:

{
  "timestamp": "2025-10-31T17:19:57Z",
  "source": "Security News Feed",
  "type": "rss",
  "data": [
    {
      "title": "Article Title",
      "link": "https://...",
      "published": "2025-10-31T12:00:00Z",
      "description": "Article description..."
    }
  ]
}

• Phase 1: Project scaffolding ✓
• Phase 2: Implement data collection from sources
• Phase 3: Save structured data for Muninn processing

Huginn outputs structured JSON data that is designed to be consumed by Muninn for further analysis, correlation, and intelligence processing.

Part of the pr-cybr rav3n-n3t framework.

This is part of the PR-CYBR ecosystem. For contributions, please follow the project guidelines.