Smart-meter data and MLOps platform
Over 1 million smart energy meters generate data and send it every 5 mins interval to Azure IoT hub.
Role: Data Architect
- smart meters
- 1M+
- smart meters
- reading interval
- 5 min
- reading interval
The approach
Over 1 million smart energy meters generate data and send it every 5 mins interval to Azure IoT hub. Azure Data Factory is used for data orchestration. Data processed through Azure IOT hub to Azure Data lake. Databricks is used for data transformation and later it is moved to SQL data warehouse and Cosmos DB. MLOps also configured which allow machine learning models to automatically train and publish endpoints through Azure ML.
Related projects
- Confidential clientMining2026
Independent audit of an enterprise Databricks Lakehouse
Led a 10-day independent audit of a global mining company's enterprise Databricks Lakehouse platform across three pillars (Ingestion, Data Governance, Technical Governance), delivering evidence-backed findings from production environment analysis of 16,000+ tables across two AWS regions and three stakeholder workshops.
- tables analysed across two AWS regions
- 16,000+
- tables analysed across two AWS regions
- remediation roadmap delivered
- 21-point
- remediation roadmap delivered
DatabricksUnity CatalogDelta LakeDelta SharingAWS (multi-region) - ChevronEnergy2025
Global data model products for the Advanced Analytics Platform
Data model products were developed for Chevron’s Data Analytics Platform to support Mobile Inspections, Engineering Documents, and Equipment management.
AzureDatabricksErwinInformaticaADF - SuncorpBanking & insurance2024
Data lake segregation rules for banking and insurance
Developed and implemented segregation rules for Suncorp's Data Lake to ensure secure and efficient handling of banking and insurance data.
AzureDatabricksMicrosoft Fabric