Javi on DataData • AI • Cloud Engineering

Temas populares

↑↓ para navegar↵ para abrirEsc para cerrar

Comunidad

De qué hablan los ingenieros de datos, y mi opinión práctica.

Sigo las discusiones de la comunidad de ingeniería de datos y reúno las preguntas, los compromisos y los problemas de producción que merecen una mirada más detenida.

Esta página es una selección, no un feed automático. Cada tema enlaza a la discusión original y añade un resumen con mis propias palabras, por qué importa y mi opinión práctica.

Pulso de la comunidad

De qué se está hablando en la comunidad de ingeniería de datos.

Los temas de la comunidad se escriben primero en inglés. Cada tarjeta indica su idioma.

  • Fuente Microsoft Fabric CommunityEn inglés

    AI-assisted development in Fabric

    This is a vendor announcement, not a community thread: Microsoft’s FabCon Europe 2026 post on the Fabric Community blog. It describes agentic Copilot in notebooks, which turns a developer’s intent into governed, Fabric-native operations, and a data engineering agent in preview, built on the Osmos acquisition. Instead of step-by-step chat, developers describe an outcome and grant access to Fabric assets; the agent plans, executes and validates work over hours or days, creating notebooks, transforming data and updating schemas on Fabric Spark. Microsoft positions it for long-running efforts such as migrations, schema harmonization and data quality remediation. These are preview capabilities as described by Microsoft; production experience is still limited.

    Mi opinión

    AI can accelerate boilerplate, exploration and repetitive implementation, but production data engineering still needs human ownership of architecture, security, schemas, performance and validation. Generated notebooks should be reviewed like any other code.

    • AI
    • LLM
    • Microsoft Fabric
    • Fabric Notebook

    Relacionado

    Discutido Añadido

    Fuente

  • Fuente RedditEn inglés

    CI/CD and Git deployment in Microsoft Fabric

    A team new to Git shared its design: workspaces per layer and environment, Dev workspaces connected to a dev branch, Test and Prod updated only by GitHub Actions using fabric-cicd, variable libraries for configuration, and SQL database projects for the Warehouse (which fabric-cicd did not deploy), with a manual review of the Warehouse diff before it runs. Report developers stay on deployment pipelines in separate workspaces. Replies challenged keeping Bronze in Prod only, since structural differences between environments tend to cause trouble later; suggested separating storage items from engineering items so feature branches can share Dev data; recommended dbt; and argued that branch policies and Git discipline matter more than the architecture. Since then, Microsoft’s August 2026 update has previewed Warehouse CI/CD with DacFx and an API for branched-workspace relationships.

    Mi opinión

    Git should be the source of truth for code and deployable artifacts, while environment-specific configuration should remain externalized. Avoid treating a Fabric workspace as the source of truth. CI/CD should promote versioned artifacts into environments rather than depend on manual workspace edits.

    • CI/CD
    • Git
    • Microsoft Fabric
    • DevOps

    Relacionado

    Otras discusiones

    Discutido Añadido

    Fuente

  • Fuente RedditEn inglés

    Delta Lake interoperability, schema and write behavior

    The question: when promoting Delta schema changes from Dev to Prod, rely on mergeSchema or overwriteSchema in the writer, or alter tables explicitly? The poster found switching mergeSchema on for one release and off again clunky. Answers differed: treat a table like a public API and never drop or retype columns (add a new column instead); keep mergeSchema on but control the columns explicitly before every write, except for Gold tables behind Direct Lake models; or route all writes through shared functions that enforce a YAML table definition. Related threads add the multi-engine side: delta-rs and Polars are fast and light for small workloads, but lag on Delta features (MERGE needed package upgrades in the Fabric runtime, Polars cannot write deletion vectors or V-Order), and one reply stated there are no plans to support Fabric-specific features in single-node libraries.

    Mi opinión

    When multiple engines can write the same Delta table, the operational contract matters more than the library choice. Be explicit about schema ownership, supported protocol/table features, evolution rules and writer compatibility. Automatic schema merging is convenient, but it should not replace deliberate schema management in production.

    • Delta Lake
    • Schema Evolution
    • Fabric Lakehouse
    • Python

    Relacionado

    Otras discusiones

    Discutido Añadido

    Fuente

  • Fuente RedditEn inglés

    Fabric architecture for a small team

    A poster shared a design for a small team: three repositories split by ownership (ingestion, analytics, reports), only Dev and Prod with no staging, Dev synced from main through Git integration, Prod promoted with deployment pipelines, a workspace pair per layer, metadata-driven Bronze, and shortcuts into a Gold workspace where Warehouse versus Lakehouse was still open. Feedback was mixed. Some warned about deployment pipelines and Warehouse deployment and suggested fabric-cicd, dbt or SQL database projects; the poster said deployment pipelines had worked so far. Others advised just starting and refactoring instead of designing everything up front, warned that capacity limits can push small teams toward larger SKUs, and offered different Gold choices depending on SQL versus Python skills and reporting performance.

    Mi opinión

    Small teams benefit from fewer moving parts. The goal should not be to copy an enterprise architecture with dozens of workspaces and environments. Separate things where ownership, risk or deployment boundaries justify it; otherwise optimize for simplicity and maintainability.

    • Data Architecture
    • Microsoft Fabric
    • Fabric Workspace
    • Medallion Architecture

    Relacionado

    Discutido Añadido

    Fuente

  • Fuente RedditEn inglés

    Moving notebooks between Dev, Test and Production

    An engineer coming from Databricks Asset Bundles asked how Fabric handles environments. Replies agree on one workspace per environment but split on promotion. Several prefer the fabric-cicd Python library run from Azure DevOps or GitHub Actions, with Git integration only on per-developer feature workspaces, and describe deployment pipelines as tedious (rules per item, rebuilding the pipeline to change stages, unclear rollback); others say deployment pipelines work well for them, and one reply notes there is no polished equivalent yet. Environment-specific values belong in variable libraries, which one person found laborious to retrofit. A related thread on notebooks gives the practical version: replace the default Lakehouse and hard-coded ABFS paths with variables or paths resolved at run time, and expect a few manual fixes the first time new items reach Test or Prod.

    Mi opinión

    Treat environment binding as configuration, not notebook logic. Notebooks should not require manual edits every time they move between Dev, Test and Production. Keep workspace-specific values externalized through variables, configuration or deployment tooling, and make environment promotion repeatable.

    • Fabric Notebook
    • Fabric Workspace
    • CI/CD
    • Git

    Relacionado

    Otras discusiones

    Discutido Añadido

    Fuente

Pregunta de la semana

Formas de participar

La comunidad es una conversación. Estas son las formas de participar a medida que estén disponibles.

  • Sugerir un tema

    Indícame una discusión o un problema de producción que merezca una mirada más detenida.

  • Hacer una pregunta técnica

    Las preguntas de proyectos reales a menudo se convierten en artículos o temas de la comunidad.

  • Contribuir a un proyecto

    Ayuda a construir los proyectos abiertos: código, documentación, ejemplos y pruebas.

  • Ayudar con las traducciones

    Mejora las versiones en francés y español de los artículos y las páginas.

  • Informar de un problema

    ¿Has visto un error, un enlace roto o un detalle desactualizado? Avísame.

  • Compartir lo que has construido

    ¿Has construido algo a partir de un artículo o proyecto del sitio? Comparte el resultado.

Los enlaces se añadirán aquí cuando cada canal esté disponible.

Lecturas sobre los temas que sigo en la comunidad.

Artículos

Proyectos

Charlas

¿Tienes un tema técnico que merezca la pena comentar?

Si una discusión de aquí se parece a un problema en el que estás trabajando, o crees que a mi opinión le falta algo, me gustaría saberlo.

Proponer un tema LinkedIn