Showing posts with label Efergy Elite. Show all posts
Showing posts with label Efergy Elite. Show all posts

Saturday, April 16, 2011

'Smart' electricity meter based on Efergy Elite monitor, part 2

Summary: In part 1, I documented the Efergy Elite’s radio protocol and wrote a PIC program that ran in a tight loop decoding the data and sending it to a serial port. This post improves upon the decoding technique used allowing for other applications (eg temperature monitoring) to multi-task on the same PIC. A simple control loop multitasking framework using interrupts to capture time critical events is explained.

The ultimate goal of this project is a electricity consumption monitor that provides real time feed back and also logs historical power use for analysis and report generation. For the moment my focus is feeding the data to a PC/laptop. In reality a more power efficient logging device will be needed. But that's for a future post.

Recently I wanted to update this receiver to include data from temperature sensors and output data from both the Efergy Elite and the temperature sensors to a single serial port.

However the tight timing loops which are continuously running make it near impossible to perform any other task. Fortunately the PIC has lots of useful hardware that can be brought to bear on this problem. Timing pulses with a loop is rather clumsy, inaccurate and wasteful of system resources. Almost all PICs have timers and interrupts which can be used for this task.

By using this peripheral hardware a lot of the decoding grunt work can be happen silently in the background, freeing up the CPU for other stuff.

Simple Multitasking with a PIC


Although not a proper multitasking processor such as a Pentium, a very simple form of multitasking can be easily achieved with a PIC. This comprises a master loop which calls application service routines for each application in turn. This is often called “simple control loop multitasking”.

The application service routines check to see if there is any work to be done. If not it immediately exits and the control loop then pass control the the next application service routine. If there is work to be done, the application service routine attempts to tackle as much as can be achieved in it’s allocated time. If all the work cannot be completed in time, it must relinquish control prematurely and continue the task on the next call.

void main (void) {

  // Initialize apps
  initAppA();
  initAppB();
  initAppC();
  initAppD();

  // Main loop
  while (1) {
    appA();
    appB();
    appC();
    appD();

    // Clear watchdog timer
    CLRWTD();  
  }
}

Architecture diagram of simple control loop multitasking with interrupts used to capture time critical events.


In order to be responsive the main loop must iterate many times a second. The iteration frequency will depend on the nature of the application(s). 1kHz is a common choice. At 1kHz the entire loop must complete in 1ms and each app service routine must take no more than 250µs (assuming there are 4 applications requiring equal CPU time).

These routines must be written so as not to use more than their allocated time slice. If a task cannot be accomplished in that time frame it must be broken into smaller work units.

Unlike a modern preemptive multitasking operating system such as Linux these constraints are not enforced. A badly written service routine will affect the performance of the entire system. For example a deadlock in one application will cause the entire system to fail. One safeguard against rogue applications is to enable the Watchdog Timer and clear it on each iteration of the loop. Should one application lock up the system will reset.

Interrupts can be used to capture time critical events. The ISR (Interrupt Service Routine) is called (almost) the instant the event occurs. Its job is to store the event for the attention of the application service routine whenever it gets control of the processor. It’s often a good idea to buffer these events in case more than one arrives before the service routine gets its turn to run.

Efergy Elite decoding


A quick recap on the Efergy Elite setup. A sensor mounted in the utility meter box transmits by 433MHz radio a power reading to the display unit every 6, 12 or 18s (configurable). On the display unit PCB I tapped into the radio base band output and connected this to a low cost PIC MCU. It does the job of decoding the signal and sending a nicely formatted record to the PC via serial IO.



Bits are encoded on the base band signal using digital FSK (Frequency Shift Keying). Three square wave cycles of 2ms duration in total (ie 1500Hz) is a logic 0 and 4 square wave cycles of 2ms duration (ie 2000Hz) is a logic 1.


Each packet is prefixed by a synchronization header of 0xAB 0xAB and 0x2D. This is followed by 8 octets (bytes) of data and one octet checksum. See previous post [link] for details.

The decoding process starts with the radio hardware which outputs the base band signal. Using pulse width measurements the symbols (bits) are decoded. First the bits are are shoved onto a 16 bit shift register until the end sync header is identified (ie the contents of the shift register will read 0xAB2D). Then 8 octets of data is read followed by one checksum octet (the checksum is the arithmetic sum of the 8 data octets).


The choice of what to implement in the ISR and what to implement in the application service routines is sometimes not clear. The rule of thumb is to keep the ISR as small and fast as possible. I had originally considered using the ISR to time pulses and send pulse durations to the application layer. As there can be as little as 250µs between pulses the entire main loop would have to execute in under that time. That’s only 250 instructions on a 4MHz clocked PIC. Not enough!

For just a few more instructions the pulse to symbol decoding can happen in the ISR thus allowing for up to 2000µs between application service routine calls.

To improve matters further, instead of storing one symbol at a time, I push the decoded symbols into a buffer (implemented as a 8 bit shift register) allowing for up to 16000µs between calls.

For the first attempt I connected the radio base band signal to the INT pin (RB0), traped INT interrupts and used Timer1 to measure the pulse widths. Timer1 is a 16 bit timer which can be clocked using the system clock divided by 4. If you’re using the 4MHz internal clock that’s a convenient 1µs per clock tick.

There are a few gotchas when using interrupts. For example: variables shared between the ISR and main program must be declared with the “volatile” modifier. When accessing state information which can be manipulated by the ISR, it is often necessary to briefly disable the interrupt while reading to ensure that an interrupt does not case the state to appear corrupted.

Measuring pulse width with INT interrupt.
The PIC can be programmed to trigger INT on a rising edge or falling edge. But not both at the same time. To measure pulse widths I’ll need both. The work around is to alternate the triggering mode.

I’m measuring inverse pulses (ie the duration of time when the signal is at low voltage). So when waiting for a pulse to arrive, set to trigger on a falling edge. In the ISR, record the clock and set trigger mode to rising edge. The next invocation of the ISR will occur at the end of the pulse. Subtract the clock from the previously recorded value -- that is the pulse width. Now reset INT to trigger on a falling edge again for the next pulse.

An alternative implementation using Capture/Compare


Microchip PICs have so much peripheral hardware that there is often several ways of solving the same problem. The Capture/Compare module is probably better suited for this problem. The INT approach does have one advantage though: if there is nothing else to do the device can be put in low power mode with the SLEEP instruction. Activity on INT will automatically wake the device up. I’ll post that implementation in a separate blog post.

The main control loop


My main loop looks like this:

// Main control loop.
 while (1) {

  // AppA: Decode Efergy Elite signal. 
  if (efergy_elite_interrupt_decode()) {

   // Have a complete record

   // Extract single phase power reading (12 bits)
   power = ((ee_buf[3]&0x0f) << 8) | ee_buf[4];

   // Display full record on the serial port
   for (i = 0; i < 8; i++) {
    writeHex(ee_buf[i]);
    putch (' ');
   }
   crlf();
  }


  // AppB: Blink LED if power use above threshold
  if (power > POWER_THRESHOLD) {

   // Cause LED to blink every 4096 iterations
   if ( (t&0x0fff) == 0) {
    DEBUG_PIN = !DEBUG_PIN;
   }
  } else {
   // LED off if below threshold
   DEBUG_PIN=0;
  }

  // Increment iteration counter
  t++;

  // Clear watchdog timer
  CLRWDT();    
 }
}

This has just two simultaneously running tasks:
  • One to decode telemetry from the Efergy Elite and write the data to the serial port
  • One to blink a LED if power levels are above some threshold
Instead of putting each application into a separate service function as in the template above, I've got blocks of code in the main loop as the code is short and simple.

The LED blinker (AppB) illustrates how to write these application service routines. A simplistic approach would be to compare power consumption against the threshold then turn on the LED, execute a delay loop for 0.5s, turn the LED off and delay again for another 0.5s. That will work, but violates the principle that the code should execute as quickly as possible (taking a whole second to complete a blink).

A smarter approach is implement LED blinking with a timer. I added an iteration counter to the main loop which acts as a low accuracy timer (it seemed easier than using the PICs built in timers). This way the code only takes a few instructions to execute.

The code which writes the record data to the serial port could be optimized in a similar way, writing just one character to the serial port per iteration. But that's an exercise for another day.

The Hardware

You will need the following:
  • Efergy Elite electricity monitor (obviously)
  • A PIC 16F627A or 16F628A (or any other PIC, but you may need to modify the code a little).
  • 1 LED
  • 1 resistor
  • A suitable PIC programmer. I use a PICKit2 from Microchip (about $30)
  • Breadboard (or other prototyping system)
  • Logic level Serial IO to PC interface (see text below)
This hack requires that you solder a wire to the base band pin or test pad of the Efergy Elite receiver unit. See my previous blog post on how to do this.  You will also need a wire for the battery ground. Beware that this will void your warranty -- so only proceed if you know what you are doing.
Schematic of Efergy Elite decoder using a PIC 16F627A or 16F628A (or equivalent).
Once upon a time interfacing a serial IO port to a PC was a straightforward if somewhat laborious task. PCs had a RS232 port which used +/- 13V levels which was not compatible with logic levels (usually 0 and 5V). You needed a level converter. A MAX232 chip + associated capacitors being one of the most popular solutions. Fortunately, nowadays the RS232 is very rarely found on modern equipment. So how do you interface serial IO at logic levels to a PC? There are many options available, and sadly may require that you purchase some hardware. But the good news is that all of these options are useful tools which can be repurposed for other projects. Here are some options:
  • FTDI cable available from SparkFun.com (DEV-09718), CoolComponents.co.uk and many others. Cost about $18.
  • The Dangerous Prototypes Bus Pirate available from SeeedStudio.com ($30 -- but you get a whole load of extra functionality with that. Well worth it.)
  • An old Nokia phone cable
  • Any development board that features a USB/Serial IO chip can probably be easily hacked for this purpose, including the Arduino, TI MSP LaunchPad etc. (Although you'd probably want to ask yourself if you are better off using the MCU on the development board instead of a separate PIC. If you want to see this decoder implemented for other MCUs leave a comment at the end of the blog post and I'll see what I can do.)

Annotated photograph of setup required to decode telemetry from a Efergy Elite electricity monitor and relay to PC.

A note on powering the PIC: as this is currently very much a prototype I haven't considered how I might power this device. Right now I'm using the PICKit2 to supply power via the ICSP (3.3 - 5V should work). Practical options are to pull the battery positive terminal out of the Efergy Elite display unit and use that to power the device. If connected to a PC via a FTDI cable, these cables usually have a +5V line which will deliver plenty of current. Or use a separate battery or wallwart power supply.

If running successfully this program will output to the serial port each record from the power sensor as it arrives (every 6, 12 or 18s). The output looks something like this:


EM 0.2.0
00 0D 5A 40 6E 00 01 00 
00 0D 5A 40 6E 00 01 00 
00 0D 5A 40 73 00 01 00 
00 0D 5A 40 6E 00 01 00 
00 0D 5A 40 6E 00 01 00 
00 0D 5A 41 BF 00 01 00 
00 0D 5A 41 BE 00 01 00 

As for implementing the temperature sensors I mentioned at the start -- that's for another day. So right now it’s not a terribly exciting application, but could form the basis of something more useful.

The code can be downloaded here. It’s written for the HI-TECH PICC compiler, and has been tested on PICs 16F627A and 16F628A. It should be easy to adapt to other PIC compilers and PIC devices, or indeed other MCU families.

Thursday, September 30, 2010

'Smart' electricity meter based on Efergy Elite monitor, part 1

Update (16 April 2011): I've got a follow-up post (part 2) which I describe a more efficient signal decoding technique using PIC interrupts and timers and allows for simple multitasking.

This series of posts describes how I hacked a low cost (€37 from Maplin) Efergy Elite electricity monitor to obtain real time power use data and to record and chart this data with the aim of understanding how electricty is used in the home and how to reduce consumption.

Describing the entire solution in one go is going to result in an excessively long post, so I'm going to break this into several parts over the next few weeks. In this post I'll describe the process involved in getting access to the raw data from the sensor unit.

The term 'smart' in the title is a misnomer. All I want to do here is record real time (at least several samples per minute) power use into a computer system. A true 'smart' meter will do telemetering, facilitate demand based tariffs, feed-in tariffs etc.

The problem with low cost energy monitors is you only get the current instantaneous usage and perhaps hourly, daily and weekly averages.

What I want is something that looks like this:


In this chart I can see my coffee machine (left on accidentally), the oven (not much I can do about that) and the fridge. From that chart I can figure the base load due to broadband router, WiFi access point, DVR etc and work out the average use of my fridge (by estimating the duty cycle and subtracting the power level while on from the base load).

So, how did I arrive at this chart?

I had an Efergy Elite electricity monitor which I received as a gift a few years ago. The product comprises a current transformer sensor which clips on to the utility meter outside. This transmits a power reading every 6 seconds over a 433MHz radio to the receiver unit indoors. The receiver has a LCD which displays instantaneous power use, and hourly, daily weekly averages (and other less interesting stuff like CO2 emissions). Photos of the sensor, receiver and internal electronics are here. There is no computer interface to tap into this data, so one needed to be hacked in.

There is a block diagram of my solution:


Looking at the receiver unit electronics, there is a test pad located near the radio part of the PCB. (The A72C01AUF chip to the bottom left of the test pad does all the radio heavy lifting both in the sensor and  receiver).


Looking at the signal at this point with an oscilloscope one can see there is burst of logic level (0/5V) signals every 6 seconds. This would seem to be the base band output from the radio.  An oscilloscope isn't the most useful tool for decoding digital IO if there are more than a few bits involved. I put a recently purchased Saleae 'Logic' (logic analyzer) on the task and this is what the signal looked like:



This signal didn't seem to make much sense. So I also simultaneously recorded the sensor's digital input line to the transmitter (pin 18 on the sensor's MCU – which by the way is a GSSP 22682X01 – a difficult chip to locate a datasheet for).


Thanks to the logic analyzer's big picture it seems there is an initial block of what looks like random noise preceding the actual data (I'm guessing this is just noise from the radio transmitter powering up).

Now looking more closely at the actual data:

The data is encoded using a digital equivalent of Frequency Shift Keying (FSK). A logic 0 is represented by three long pulses of 2ms duration in total (a low frequency) and a logic 1 by 4 shorter pulses totaling 2ms also (a high frequency).

Decoding this could be achieved with analog circuitry but this would involve a handful of (relatively) expensive passive components. Instead I chose a pure digital technique. I used a simple pattern matching algorithm to look for 3 successive long pulses or 4 successive short pulses and ignore anything else (code appended below).

From bits to bytes

So now I have a string of logic '0's and '1's. The next problem is to determine where the byte boundaries are.

Fortunately each data packet starts with the following 4 bytes: 0xAB 0xAB 0xAB 0x2D. This helps with synchronization. When looking for the start of data I shove the bits into a shift register until I get a byte value of 0xAB. I take all subsequent bits as groups of 8 to form the bytes of data. After the first 0xAB I ignore all subsequent byte values of 0xAB until I get a 0x2D. I am now at the start of the packet payload.

A typical packets look like this:
AB, AB, AB, 2D, 00, 0B, 5A, 40, 98, 00, 02, 00, 41
After some experimentation this is what I believe the packet consists of:

0Sync, always 0xAB
1Sync, always 0xAB
2Sync, always 0xAB
3Sync end,  always 0x2D
4? always 0x00
5Device address high byte (always 0x0D for my device)
6Device address low byte  (always 0x5A for my device)
7sbssaaaa where s = bits of sampling period, b = battery low indicator (0 = low) and aaaa is the upper 4 bits of sensor A. 0-00---- = 6s sampling, 1-01---- = 12s sampling, 0-10---- = 18s sampling. It's odd that ‘b’ is sandwiched in among s bits.
8Least significant 8 bits of sensor input A (nearest MCU edge of board)
9? probably upper 4 bits of sensor B and C.
10Least significant 8 bits of sensor input B (middle)
11Least significant 8 bits of sensor input C
12Checksum (arithmetical sum of bytes 4 - 11).


So in summary, each packet comprises a unique sensor address, the sampling rate, a battery low flag and 3 x 12 bit ADC values for each of the sensor's three inputs  (normal domestic power is single phase, so only one input is used. A three phase system would use all three inputs) and finally a checksum of the contents of the packet for data integrity.

I soldered a small (12F675) PIC MCU to a piece of veroboard (pictured right) and wrote a program to decode the baseband signal from the radio and convert that to actual data.

The PIC code used to decode the radio baseband signal is listed below (this excerpt is not the full PIC application – contact me if you need the lot):

The next problem is to figure how to translate these ADC values to Amps (and by multiplying by 230V to Watts) and how to feed this data to a database for recording and visualization. These will be covered in subsequent posts.

Related Links
Updates to post:


#include 
#include "efergy_config.h"
#include "efergy_elite_decode.h"


byte getPulseWidth();
byte getBit ();
byte getByte ();
void resync();

byte t3,t4,tm;


/**
 * Decode baseband radio signal from Efergy Elite receiver.
 * 
 * Packet consists of a radio preamble of what seems
 * like random noise for about 200ms followed by a quiet period
 * of 100ms followed by data for 200ms. Encoding is a FSK type
 * scheme.
 * Logic 0 is represented by 3 cycles of "low" frequency square 
 * wave (period 8t per cycle). 
 * Logic 1 is represented by 4 cycles of "high" frequncy square
 * wave (period 6t per cycle).
 * Logic 0 and Logic 1 have the same period of 24t which is
 * very approx 2ms [confirm]. Ie t ~= 83 microseconds.
 *
 *    +---+   +---+   +---+
 *    |   |   |   |   |        = Logic 0
 * +--|   +---+   +---+
 *   +--+  +--+  +--+  +--+
 *   |  |  |  |  |  |  |       = Logic 1
 * +-+  +--+  +--+  +--+
 *
 * Decoding this not but looking at frequency, but looking
 * for 3 long pulses in succession or 4 short pulses. 
 * 
 * Data has 3 bytes of synchronization: 0xAB, 0xAB, 0x2D.
 * Look initially for 0xAB to get byte boundary synchronization. 
 * Then look for 0x2D to flag start of actual data.
 *
 * All inter-pulse calculation must be complete 
 * within 64x5 microseconds ie ~ 300 microseconds or 300 instructions @ 4MHz)
 */

byte decodeEfergy(unsigned char *buf) {
 byte i,bitc=0, bytec=0, b=0;
 byte t,pt;
 //byte bufIndex=0;
 unsigned int rt=0;

 t3=0x1a; // short pulse duration of 3t
 t4=0x23; // long pulse duration of 4t
 tm = 0x1e; // mean of short and long pulse, ie 3.5t

resync:

 // Look for sync bit sequence
 // TODO: update: just look for AB followed by 2D.

 // Get byte boundary sync
 while (b != 0xAB) {
  b = (b<<1) | getBit();
 }
 for (i = 0; i < 8; i++) {
  b = (b<<1) | getBit();
 }
 if (b != 0xAB) {
  goto resync;
 }

 for (i = 0; i < 8; i++) {
  b = (b<<1) | getBit();
 }
 if (b != 0x2D) {
  goto resync;
 }


 b=0;

 // 9 bytes left to retrieve

 do {
  b = (b<<1) | getBit();

  bitc++;
  if (bitc == 8) {
 
   buf[bytec++] = b;
   b=bitc=0;
  }
 } while (bytec < 9);

 // Calculate checksum
 byte cs = 0;
 for (i = 0; i < 8; i++) {
  cs += buf[i];
 }

 // Display checksum fail if they don't match
 if (cs != buf[8]) {
  return 4;
 }

 return 0;
}

byte getByte () {
 byte i,b;
 for (i = 0; i < 8; i++) {
  b = (b<<1) | getBit();
 }
 return b;
}

byte getBit () {
 char t;
 byte nshort=0,nlong=0;

 while (1) {
 
  t = getPulseWidth();

  if ( t < tm) {
   nshort++;
  } else {
   nlong++;
  }

  // Check for anomalous condition
  if ( (nshort>0) && (nlong>0) ) {
   nshort=nlong=0;   
   if ( t < tm) {
    nshort=1;
   } else {
    nlong=1;
   }
  }

  if (nshort == 4) {
   return 1;
  }
  if (nlong == 3) {
   return 0;
  }
 }
}

byte getPulseWidth() {
 byte t=0;
 while (RADIOBB_PIN) {
  CLRWDT();
 }
 while ( ! RADIOBB_PIN) {
  t++;
 }
 return t;
}

void resync () {

 byte pt,t;

 restart_resync:

 // Can have either 3t (short) or 4t (long) pulse. Get pulses of
 // two different lengths. 3t is the short pulse, 4t is the long one.
 pt = getPulseWidth();
 do {
  t = getPulseWidth()+2;
 } while (t - pt < 4);
 t -= 2;


 if (pt < t) {
  t3 = pt;
  t4 = t;
 } else {
  t3 = t;
  t4 = pt;
 }

 if (t4 < 0x1A) {
  //TXREG='s';
  goto restart_resync;
 }
 if (t4 > 0x30) {
  //TXREG='l';
  goto restart_resync;
 }

 // Didn't have to do this at the sensor, but receiver has a lot more incoming crud
 if ( ((t4>>2)*3 ^ t3) & 0b11111100 ) {   // (t4/4)*3  ~= t3 (ignoring 2 LSB of precision)
  //TXREG='?';
  goto restart_resync;
 }
 

 // tm is mean of t3 and t4
 tm = (t3+t4)>>1;

 byte b=0;

 // Look for sync bit sequence
 while (b != 0xAB) {
  b = (b<<1) | getBit();
 }
 
}
  翻译: