DSMR to MQTT

Dutch smart meter to MQTT without external power supply.



Created by: Max Engelen, 2020-04-10


Currently, lots of homes in the Netherlands switch from regular power consumption meters to so called ‘slimme meter’. These devices do not only record the consumption of electrical power, but also allow these measurements to be communicated to the electricity supplier for monitoring and billing. These ‘slimme meters’ also feature a P1 interface which spits out serial updates that allows reading your energy consumption in real-time(± 10s interval).

The serial updates from the P1 interface are called ‘telegrams’, an example of such a telegram would be as stated below although the exact format depends on the version of your meter. The values of interest are marked with //explanation. This is not part of the message. What all the other values are and which version your meter runs is nicely explained in the documentation or on this website: http://domoticx.com/p1-poort-slimme-meter-hardware/

  
/XMX5LGBBFG10
 
1-3:0.2.8(42)  
0-0:1.0.0(170108161107W)  
0-0:96.1.1(4530303331303033303031363939353135)  
1-0:1.8.1(002074.842*kWh)             //power usages low rate  
1-0:1.8.2(000881.383*kWh)             //power usages high rate  
1-0:2.8.1(000010.981*kWh)             
1-0:2.8.2(000028.031*kWh)  
0-0:96.14.0(0001)  
1-0:1.7.0(00.494*kW)                   //curent power usage 
1-0:2.7.0(00.000*kW)            
0-0:96.7.21(00004)
0-0:96.7.9(00003)
1-0:99.97.0(3)(0-0:96.7.19)(160315184219W)(0000000310*s)(160207164837W)(0000000981*s)(151118085623W)(0000502496*s)
1-0:32.32.0(00000)
1-0:32.36.0(00000)
0-0:96.13.1()
0-0:96.13.0()
1-0:31.7.0(003*A)
1-0:21.7.0(00.494*kW)
1-0:22.7.0(00.000*kW)
0-1:24.1.0(003)
0-1:96.1.0(4730303139333430323231313938343135)
0-1:24.2.1(170108160000W)(01234.000*m3)   //gas usage
!D3B0  //Checksum

 >

Hardware

We need to have a device that is capable of receiving these messages and sending the data into the cloud using MQTT. The ESP8266 is the best option for me at the moment although this will probably change with the release of the ESP32-S2 somewhere in the summer of 2020. For now, we chose the esp-01 because of its amazing form factor and extreme low power in deep sleep mode (only available with some DIY soldering (https://www.instructables.com/id/Enable-DeepSleep-on-an-ESP8266-01/) on the board, go with an esp-12f if this is not your thing).

Next up: the p1 connection. You will need a p1 cable with 6 conductors for this project. The pinout follows this table:

pin# Signal name Description
1 + 5v Power supply
2 Request input
3 data GND ground for data signal
4 N.C. not connected
5 Data Data output
6 power Gnd Power supply ground

The DSMR 4.2 specifications, to which all Dutch smartmeters must comply, rate the 5v power supply (pin 1 of the P1 port) at 100mA. The ESP8266 most of the time draws between 70 and 80 mA. However, during startup, there are noticeable spikes above 100mA (even my multimeter shows them) and occasionally (during transmit) it also spikes above 100 mA.

Andreas Speiss has a nice youtube video on how to solve this: https://www.youtube.com/watch?v=6SdyImetbp8&t=230s. Suggesting a 2200uF capacitor behind your voltage regulator. When choosing your voltage regulator, go with a linear one from 5v to 3.3v because 5v will fry your esp board. (If there is an available power outlet near your DSRM-reader, you can choose to use an external power supply to avoid the current-draw issue. I don’t have an extra power outlet near my DSMR-reader and I find using the power supplied by the P1 port a way more elegant solution).

The P1 port only sends data if the RTS pin has> + 5v relative to the GND (-), as long as the voltage remains on it, a “telegram” is sent every 10 seconds on the data pin. With the following serial specs for version 4:

DSMR 4.0/4.2
115200 8N1:
Baudrate = 115200
Data bits = 8
Parity = None
Stop bits = 1

Weird thing is that they inverted the data bus so we need to apply an extra transistor to revert it.

Putting it all together gives us the following design plus some debug options:

Software

The esp firmware will communicate the power consumption details to an MQTT server which means you need to have an MQTT server running somewhere in your network. I myself use hassio (https://www.home-assistant.io/docs/mqtt/broker/) which is a home automation hub implemented on a raspi. Hassio manages the MQTT broker, puts the data in influxdb and displays the data to me using grafana.

A nice feature to add in the future might be adding a crc16 redundancy to validate the data.

The Arduino IDE compatible code is included in the downloadable files at the bottom of this page. It depends on the following libraries:

  • ESP8266WiFi.h
  • pubSubClient.h
  • ESP8266WebServer.h
  • ESP8266mDNS.h
  • ESP8266HTTPUpdateServer.h

Conclusion

Everything works. I implemented it DIY style but hope to make a PCB with some more functionalities later. Status now:

And how I display the data in grafana:

If you have any questions please ask me.

Lastly: if you just want to read the serial output from the p1 port with a PC or Raspberry, this cable will help: https://www.bol.com/nl/p/slimme-meter-kabel-p1-usb/9200000111535827/