DEV Community

#dataengineering

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
How AI can investigate a 12 GB export without reading it into a prompt
jerry_king_d0ee94a9bca82f profile

How AI can investigate a 12 GB export without reading it into a prompt

#dataengineering #sql #ai #parquet
Comments
6 min read
How I Built a Two-Tier Data Platform in 9 Phases (Spark, Kafka, dbt, Kubernetes)
adnanphp profile
Adnan

How I Built a Two-Tier Data Platform in 9 Phases (Spark, Kafka, dbt, Kubernetes)

#dataengineering #spark #kafka #kubernetes
Comments
6 min read
Data apps, defined: levels, principles, and a scorecard builders can actually use
sangaraju1988 profile

Data apps, defined: levels, principles, and a scorecard builders can actually use

#dataengineering #ai #analytics #architecture
5
Comments 4
8 min read
The AI Loop That Won't Let You Go - Victor Amit
victoramit profile

The AI Loop That Won't Let You Go - Victor Amit

#ai #machinelearning #dataengineering #opensource
1
Comments
20 min read
Auto CDF Moved the CDC Tax From Writes to Reads: Delta CDF vs Auto CDF vs Spark 4.2 CHANGES
firfircelik profile

Auto CDF Moved the CDC Tax From Writes to Reads: Delta CDF vs Auto CDF vs Spark 4.2 CHANGES

#dataengineering #databricks #spark #deltalake
4
Comments 5
9 min read
I turned Brazil's 73M-establishment company registry into second-fast lookups, with no server and no database
jeffev profile

I turned Brazil's 73M-establishment company registry into second-fast lookups, with no server and no database

#duckdb #python #opendata #dataengineering
Comments
3 min read
EU SaaS Metrics Dashboard: Privacy-Friendly Product KPIs and Backend Counter Reconstruction
fluxh91 profile

EU SaaS Metrics Dashboard: Privacy-Friendly Product KPIs and Backend Counter Reconstruction

#observability #privacy #dataengineering
Comments
7 min read
Opening a Parquet file in the browser without losing decimals, big integers or timestamps
vedaforge_eng profile

Opening a Parquet file in the browser without losing decimals, big integers or timestamps

#parquet #dataengineering #javascript #data
1
Comments
2 min read
Context, Not Models: What Actually Made AI BI Reliable
datatechbridge profile

Context, Not Models: What Actually Made AI BI Reliable

#ai #dataengineering #llm #sql
Comments
25 min read
Building a Data Audit Workbench That Holds Up in an Assessment
maref profile

Building a Data Audit Workbench That Holds Up in an Assessment

#compliance #dataengineering #audit
Comments
3 min read
Building a Data Audit Workbench That Holds Up in an Assessment
maref profile

Building a Data Audit Workbench That Holds Up in an Assessment

#compliance #dataengineering #audit
Comments
3 min read
A bad batch loaded with zero errors. One row added $4.5 million.
jigonyoo profile

A bad batch loaded with zero errors. One row added $4.5 million.

#dataengineering #dbt #dataquality #python
Comments
5 min read
Best Snowflake Data Modeling Tools: SQLDBM, erwin, Hackolade & Vertabelo for Reverse Engineering and DDL Generation
naveensub profile
Naveen

Best Snowflake Data Modeling Tools: SQLDBM, erwin, Hackolade & Vertabelo for Reverse Engineering and DDL Generation

#database #dataengineering #sql #tools
Comments
11 min read
Data Engineering Failures Cost Enterprises $3M a Month
datadriven profile

Data Engineering Failures Cost Enterprises $3M a Month

#dataengineering #career #devops #database
Comments
8 min read
Counting a phrase across the King James Bible — and why "fear not" is not 365 times
makeinterview profile

Counting a phrase across the King James Bible — and why "fear not" is not 365 times

#data #dataengineering
Comments
3 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.