data-pipeline — independent software & tools

9 independent data-pipeline listings the big search engines bury.

• news-briefing

  github · Free · by elanthus

  Auditable daily news briefing with a code-enforced source window, model-ranked stories, and structural corpus checks.

• brokoli

  github · Free · by Tnsor-Labs

  Brokoli — self-hosted data pipeline orchestration

• rag-ready-extractor

  github · Free · by delilahsaprophytic338

  Convert webpages and PDFs into clean, structured data optimized for retrieval-augmented generation in a single API call.

• drt

  github · Free · by drt-hub

  Reverse ETL for the code-first data stack

• faucet-stream

  github · Free · by faucet-hq

  The fast, config-driven way to move data in Rust — a native CLI and an embeddable Rust ETL library

• Winmem-Witness

  github · Free · by kazuop

  🌐 Preserve Solana projects effortlessly with Winmem, a self-hostable AI runtime that maintains verifiable records even when teams step away.

• cbb-ratings-pipeline

  github · Free · by kylevogel

  End-to-end Python ETL pipeline that collects NCAA men's basketball rankings from multiple sources, standardizes team names, and deploys a live dashboard via GitHub Pages.

• doc2dataset

  github · Free · by reisel-g

  📄 Ingest documents into structured datasets for LLMs, ensuring numeric integrity and easy export across multiple frameworks with doc2dataset.

• pyspark-etl-automation

  github · Free · by roseannspastic496

  🚀 Automate ETL processes with PySpark to transform and load transactional data into PostgreSQL for efficient analytics.