DEV Community

#dataengineering

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
Netflix Conductor 4.0: Scaling Workflows to 30,000 Tasks with Asynchronous Evaluation
aidatalegends profile

Netflix Conductor 4.0: Scaling Workflows to 30,000 Tasks with Asynchronous Evaluation

#aiinfrastructure #dataengineering #distributedsystems #scalability
Comments
3 min read
Moving Big Data from GCP to AWS: BigQuery, Dataproc and Dataflow
emmanuelldev profile

Moving Big Data from GCP to AWS: BigQuery, Dataproc and Dataflow

#aws #googlecloud #dataengineering #bigdata
Comments
16 min read
Salesforce to Snowflake, With Less Friction: Zero-Copy V2 in Practice
swaroop_krishna_e2f4b83b2 profile

Salesforce to Snowflake, With Less Friction: Zero-Copy V2 in Practice

#snowflake #salesforce #dataengineering #zerocopy
Comments
5 min read
Monitoring AI Citations at Enterprise Scale: What to Track (and What the Data Can't Tell You)
subin_tpsubintp_4874b8f profile

Monitoring AI Citations at Enterprise Scale: What to Track (and What the Data Can't Tell You)

#ai #webdev #seo #dataengineering
Comments 1
4 min read
Spark JDBC Parallel Reads on Azure Databricks: partitionColumn, Bounds, Predicates, and Not Knocking Over Your Database
karanpulyala profile

Spark JDBC Parallel Reads on Azure Databricks: partitionColumn, Bounds, Predicates, and Not Knocking Over Your Database

#databricks #pyspark #spark #dataengineering
1
Comments
5 min read
Data Lifecycle & Storage Tiering for Cost Efficiency
beefedai profile

Data Lifecycle & Storage Tiering for Cost Efficiency

#dataengineering
Comments
8 min read
Building an Enterprise AI Visibility Monitoring Pipeline: Prompt Sets, Logging Schema and Dashboards
rida_moideen profile

Building an Enterprise AI Visibility Monitoring Pipeline: Prompt Sets, Logging Schema and Dashboards

#dataengineering #enterprisemarketing #singaporetech #geo
Comments
4 min read
Debezium Postgres to Kafka: Every Setting Explained
pipelinewhiteboard profile

Debezium Postgres to Kafka: Every Setting Explained

#dataengineering #postgres #kafka #tutorial
Comments
8 min read
OCR is a step, not a guarantee (turning scanned archives into usable data)
fabienk profile

OCR is a step, not a guarantee (turning scanned archives into usable data)

#ocr #automation #dataengineering #productivity
Comments
5 min read
Your CSV importer needs an exception report, not just column mapping
mapryva profile

Your CSV importer needs an exception report, not just column mapping

#python #csv #dataengineering #abotwrotethis
Comments
5 min read
Dataflow Gen2 Eating Your Microsoft Fabric Capacity? Why Default Settings Burn 60-80% Extra CUs
costfabric profile

Dataflow Gen2 Eating Your Microsoft Fabric Capacity? Why Default Settings Burn 60-80% Extra CUs

#microsoftfabric #dataengineering #powerbi #etl
Comments
3 min read
Power BI Premium P1 to Microsoft Fabric F64 Migration: Why Most Teams Are Over-Provisioned
costfabric profile

Power BI Premium P1 to Microsoft Fabric F64 Migration: Why Most Teams Are Over-Provisioned

#powerbi #microsoftfabric #finops #dataengineering
Comments
3 min read
Metadata Graph for Data Engineering: A Practical Guide
agenticstacks123 profile

Metadata Graph for Data Engineering: A Practical Guide

#dataengineering #ai #metadata #tutorial
Comments
11 min read
1% of Layoffs Blamed on AI: What Gallup Means for Data Engineers
datadriven profile

1% of Layoffs Blamed on AI: What Gallup Means for Data Engineers

#dataengineering #career #ai #interview
Comments
8 min read
Why Pausing Your Microsoft Fabric Capacity Triggers Surprise Azure Bills (Smoothing & Carryforward)
costfabric profile

Why Pausing Your Microsoft Fabric Capacity Triggers Surprise Azure Bills (Smoothing & Carryforward)

#microsoftfabric #powerbi #dataengineering #azure
1
Comments 1
4 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.