data-engineering — independent software & tools
-
Weather-DataOps-Pipeline
Automated weather data pipeline demonstrating CI/CD, Docker, GitHub Actions and DataOps principles.
-
tech-brief
Daily tech-watch digest (AI, dev, DevOps, data, cloud). Auto-published from my watch on baptisteblouin.fr
-
datacharter
Your data, explored locally — and your AI agents kept on a leash. A federated data explorer with governed agentic access.
-
csvql
Local-first CSV analytics CLI powered by DuckDB — query, profile, validate, catalog, and export local CSVs.
-
github-insight-radar
GitHub repository intelligence radar with audience-based scoring, quality gates, reports and dashboard-ready outputs.
-
maestro
Orchestrate AI coding agents with plain markdown files, no setup, no dependencies, and support for Claude Code, Gemini, Codex, and Cursor
-
brokoli
Brokoli — self-hosted data pipeline orchestration
-
swath
Parallel, resumable listing for very large S3 buckets. Learns the key distribution as it runs, streams JSONL or TSV, and writes crash-resumable Parquet you can query directly with DuckDB.
-
sqrail
A bounded SQL-on-files CLI for coding agents. SQL in, files out.
-
vajra
Break noise. Preserve truth. A deterministic semantic reduction engine for structured data — JSON, YAML, CSV, NDJSON, Markdown, PDF. Fingerprinting, entropy, anomaly detection, schema drift, clustering, and AI-ready essence generation. Built in Rust.
-
drt
Reverse ETL for the code-first data stack
-
faucet-stream
The fast, config-driven way to move data in Rust — a native CLI and an embeddable Rust ETL library
-
odctl
A curated collection of open source technologies and an accompanying CLI for experimenting with modern data architecture and MLOps.
-
market-data-platform
Python-based financial market data platform for ingestion, validation, transformation, storage, and analysis of futures, equities, macro data, and time-series datasets.
-
dota2-liveops-platform
Automated ELT pipeline tracking Dota 2 Pro Meta. Built with Python, Postgres, and GitHub Actions.
-
Project_Synapse
🧠 Build a cutting-edge AI agent with Project Synapse that enables strategic thought and multi-turn collaborative dialogue for innovation and discovery.
-
pyspark-etl-automation
🚀 Automate ETL processes with PySpark to transform and load transactional data into PostgreSQL for efficient analytics.
-
atlas
Atlas — self-hosted, source-configurable, multi-disciplinary engineering platform for gen-AI, ML, and data work. 30+ OSS Docker Compose services (Ollama, LiteLLM, Supabase, n8n, Airflow, Spark, Weaviate, Neo4j, JupyterHub, Ray, ComfyUI, …) pre-wired behind Kong. Interactive TUI wizard, configurable tracks, one-command launch.