Everything About: Dataengineer

👋 Sign in for the ability to sort posts by relevant, latest, or top.

Rupak Kulkarni

Nov 2

A Green Spark Job Isn't Enough: Building a Fail-Closed Pipeline Health Monitor in Python

#python #dataengineering #devops #testing

6 min read

Aqib Javaid

Nov 2

How Much Does a Custom Web Scraper Cost to Build?

#webscraping #dataengineering #datapipelines #softwareestimation

6 min read

Aqib Javaid

Nov 2

Web Scraping vs Official API: Which Should You Use?

#webscraping #apiintegration #dataengineering #architecture

6 min read

Kennedy Njoroge

Nov 2

Equality deletes have no address, and that's why your Iceberg reads crawl

#dataengineering #iceberg #sql #flink

7 min read

Mohammed Arshad Ansari

Nov 2

Iceberg vs Parquet: do you need a table format?

#dataengineering #database #data #analytics

3 min read

Tan Tran

Nov 1

Proving exactly-once by killing the stream mid-backlog

#dataengineering #spark #iceberg #kafka

8 min read

K HUNNURJI RAO

Nov 1

Spark Is Lazy, and That One Fact Explains Most Beginner Confusion

#pyspark #dataengineering #beginners #python

5 min read

Dataspryva

Nov 1

Your premium preview audit may be measuring a probe cap

#datascience #python #analytics #dataengineering

7 min read

Abel Fernando PACOMPIA ORTIZ

Nov 1

Un pipeline de clima del Perú con dbt, DuckDB y GitHub Actions, sin AWS

#dataengineering #python #dbt #githubactions

6 min read

Pipeline Whiteboard

Oct 31

CDC Schema Changes: What Breaks and How to Ship Them

#dataengineering #kafka #snowflake #database

7 min read

Raj Gahoi

Oct 31

Bridging Python and Rust: Building Type-Safe Data Quality Pipelines with PyO3 and JSON Schema 2020-12 #rust #python #pyo3 #jsonschema

#dataengineering #performance #python #rust

9 min read

Pranjul Chaurasiya

Oct 31

Bridging the Lakehouse: Ingesting Databricks Unity Catalog into Cognee Knowledge Graphs

#databricks #ai #dataengineering #python

4 min read

Aniket Abhishek Soni

Oct 31

Don't treat Fabric data ingestion as a one-size-fits-all problem

#microsoftfabric #dataengineering #analytics #azure

5 min read

Muhammad Umair Ashraf

Oct 31

AWS + Snowflake: Building Modern Data and AI Workflows with Glue, S3, Iceberg, and Cortex

#aws #snowflake #dataengineering #cloud

7 min read

Pipeline Whiteboard

Oct 30

CDC Deletes, Tombstones and the Idempotent MERGE

#dataengineering #kafka #snowflake #sql

7 min read

👋 Sign in for the ability to sort posts by relevant, latest, or top.