Migrate from ELK Stack

Easy 3-6 hours

Simplify your logging infrastructure by replacing the complex ELK (Elasticsearch, Logstash, Kibana) stack with LogWard's all-in-one solution. No more cluster management or version compatibility issues.

Why Migrate from ELK?

Dramatically Simpler

No more managing Elasticsearch clusters, Logstash pipelines, and Kibana. LogWard is a single Docker Compose deployment.

Lower Resource Usage

Elasticsearch requires significant RAM (heap size). LogWard with TimescaleDB uses resources more efficiently with better compression.

Built-in SIEM

ELK requires additional components (SIEM, Security) for threat detection. LogWard includes Sigma rules and incident management.

No Version Headaches

ELK components must be version-matched. LogWard is a single versioned release with all components guaranteed compatible.

Feature Comparison

Feature	ELK Stack	LogWard
Components	3+ (ES, Logstash, Kibana)	Single stack
Log Ingestion	Beats, Logstash	HTTP API, SDKs, OTLP
Query Language	Lucene / KQL	REST API + Full-text
Full-text Search	Yes	Yes
Real-time Streaming	Kibana Discover	SSE
Alerting	Watcher / ElastAlert	Built-in
Security Detection	Elastic SIEM (paid)	Sigma (included)
OpenTelemetry	APM Server	Native OTLP
Cluster Management	Complex (shards, replicas)	Simple (PostgreSQL)
Memory Requirements	High (ES heap: 16-32GB)	Moderate (4-8GB)
Pricing	Open-source + paid features	Fully open-source

Step 1: Inventory Your ELK Setup

Document your existing ELK configuration:

What to Document

Data shippers: Filebeat, Metricbeat, Logstash pipelines
Indices: List indices and their mappings
Logstash pipelines: Document filter/output configs
Kibana dashboards: Export saved objects
Watcher/alerts: Document alert configurations

Export Kibana saved objects:

bash

# Export all saved objects from Kibana
curl -X POST "http://kibana:5601/api/saved_objects/_export" \
  -H "kbn-xsrf: true" \
  -H "Content-Type: application/json" \
  -d '{
    "type": ["dashboard", "visualization", "search"],
    "includeReferencesDeep": true
  }' > kibana_export.ndjson

Step 2: Deploy LogWard

See the Deployment Guide. LogWard requires far fewer resources than ELK:

ELK Stack Requirements

Elasticsearch: 16-32 GB RAM (heap)
Logstash: 4-8 GB RAM
Kibana: 2-4 GB RAM
Total: 22-44 GB RAM minimum

LogWard Requirements

Backend: 2-4 GB RAM
TimescaleDB: 4-8 GB RAM
Redis: 1 GB RAM
Total: 7-13 GB RAM

bash

# Clone and start LogWard
git clone https://github.com/logward-dev/logward.git
cd logward/docker
cp .env.example .env
docker compose up -d

# Verify
curl http://localhost:8080/health

Step 3: Replace Beats/Logstash

Replace Filebeat/Logstash with Fluent Bit or direct SDK integration:

Filebeat to Fluent Bit

Before (Filebeat)

yaml

# filebeat.yml
filebeat.inputs:
  - type: log
    paths:
      - /var/log/app/*.log

output.elasticsearch:
  hosts: ["elasticsearch:9200"]
  index: "app-logs-%{+yyyy.MM.dd}"

After (Fluent Bit)

ini

# fluent-bit.conf
[INPUT]
    Name tail
    Path /var/log/app/*.log
    Tag app

[OUTPUT]
    Name http
    Match *
    Host logward.internal
    Port 8080
    URI /api/v1/ingest
    Format json
    Header X-API-Key lp_xxx

Logstash Pipeline to Fluent Bit

Before (Logstash)

ruby

# logstash.conf
input {
  beats { port => 5044 }
}

filter {
  grok {
    match => { "message" => "%{TIMESTAMP_ISO8601:time} %{LOGLEVEL:level} %{GREEDYDATA:msg}" }
  }
}

output {
  elasticsearch {
    hosts => ["elasticsearch:9200"]
    index => "app-%{+YYYY.MM.dd}"
  }
}

After (Fluent Bit)

ini

# fluent-bit.conf
[INPUT]
    Name forward
    Listen 0.0.0.0
    Port 24224

[FILTER]
    Name parser
    Match *
    Key_Name message
    Parser app_log

[OUTPUT]
    Name http
    Match *
    Host logward.internal
    Port 8080
    URI /api/v1/ingest
    Format json
    Header X-API-Key lp_xxx

Step 4: Query Migration (Elasticsearch to LogWard)

Elasticsearch Query DSL translates to LogWard REST API parameters:

Elasticsearch Query	LogWard API
`{"match": {"service": "api"}}`	GET /api/v1/logs?service=api
`{"match": {"level": "error"}}`	GET /api/v1/logs?level=error
`{"query_string": {"query": "timeout"}}`	GET /api/v1/logs?q=timeout
`{"range": {"@timestamp": {"gte": "now-1h"}}}`	GET /api/v1/logs?from=2025-01-15T11:00:00Z
`{"aggs": {"by_service": {"terms": {"field": "service"}}}}`	GET /api/v1/logs/aggregated?interval=1h

KQL (Kibana Query Language) Translation

KQL	LogWard
service: api	?service=api
level: error OR level: critical	?level=error&level=critical
"connection timeout"	?q=connection%20timeout
service: api AND level: error	?service=api&level=error

Step 5: Alert Migration

Convert Elasticsearch Watcher or ElastAlert rules to LogWard alert rules:

Elasticsearch Watcher

json

{
  "trigger": {
    "schedule": { "interval": "5m" }
  },
  "input": {
    "search": {
      "request": {
        "indices": ["app-*"],
        "body": {
          "query": {
            "match": { "level": "error" }
          }
        }
      }
    }
  },
  "condition": {
    "compare": {
      "ctx.payload.hits.total": { "gt": 100 }
    }
  },
  "actions": {
    "email_admin": {
      "email": {
        "to": "admin@example.com"
      }
    }
  }
}

LogWard Alert Rule

json

{
  "name": "High Error Rate",
  "enabled": true,
  "level": ["error"],
  "threshold": 100,
  "timeWindow": 5,
  "emailRecipients": [
    "admin@example.com"
  ]
}

Concept Mapping

ELK Term	LogWard Equivalent	Notes
Index	Project	One index pattern = One project
Document	Log entry	1:1 mapping
Field	metadata key	Store custom fields in metadata JSON
@timestamp	time	ISO 8601 format
Filebeat	Fluent Bit / SDK	Use Fluent Bit for file tailing
Logstash	Fluent Bit / SDK	Use Fluent Bit filters or preprocess in app
Kibana	LogWard UI	Built-in web interface
Watcher	Alert Rules	Simpler configuration
Elastic SIEM	Sigma Rules + SIEM Dashboard	Included at no extra cost

Common Issues

Complex Logstash filters

If you have complex grok patterns or ruby filters, consider:

Moving parsing to your application (emit structured JSON)
Using Fluent Bit parsers for simple patterns
Using OpenTelemetry Collector for advanced transformations

Kibana dashboard recreation

LogWard doesn't have Kibana-style drag-and-drop dashboards yet. Use the SIEM dashboard for security metrics, and the Query API for custom integrations with tools like Grafana.

Index lifecycle management

Elasticsearch ILM is replaced by TimescaleDB retention policies. Configure retention per-project in LogWard settings. Compression is automatic.

Next Steps

API Reference

OpenTelemetry Integration

Deployment Guide