New data shield

My mum and dad bought me an Arduino Data Shield for Christmas (from Proto-Pic) but due to the weather and the fact if was sourced from America it did not arrive until after Christmas (over 3 weeks actually). I was initially surprised at the size of the package when it finally arrived and then discovered it was a kit of parts and not an assembled item (not a problem but I didn't spot this on the web site where it was bought from although looking at it again it does make this clear):

Arduino Data Logger Shield Parts

I followed the instructions on the packet of parts at Logger Shield and it took around an hour to solder all the parts onto the PCB. A few notes:

  1. The instructions seem to omit the soldering of the 6pin header although it was obvious where it goes.
  2. Having spent many an hour desoldering chips from boards when they fail I prefer to use sockets. The kit comes with an 8 pin socket for the DS1307 real time clock (which it suggests you don't use) but does not come with a 14 pin DIL socket for the (74AHC125) Level shifter for the SD card. In both cases I soldered a socket in place and later pushed the chip into the socket.
  3. You'll need a fairly small solder tip to solder the SD card holder as 2 of the pins attach to rather small pads on the PCB.
  4. The kit comes with a 36 pin male header to connect the shield to the Arduino but no equivalent sockets to replicate those on the Arduino (although the PCB contains holes for them). This was rather disappointing as my current project is wired up to a bread board and after inserting the data shield there is no where to push the wires from the breadboard into the Arduino/Data Shield (as no sockets are replicated on the Data Shield). I initially got around this by soldering a few female header pieces (for ICs) into the duplicated holes in the PCB but long term I'd replicate the female headers from the Arduino on the Data Shield. If you look carefully in the following picture you can see them for GND and +5V and D3 to D6. This is a little disappointing and if the manufacturer is reading this would be a simple and cheap but useful addition.

My finished Data Logger plugged in to a Arduino in an Arduino plastic case (I also got for Christmas purchased from Cool Components) is pictured below:

Arduino Data Logger Shield Completed

The case contains 3 plastic stands moulded to the base which the Arduino pushed over aligned through holes in the Arduino PCB. They are so tall they didn't allow the Data Shield to be pushed over the Arduino when in the box as there are no equivalent holes in the Data Shield PCB so I was forced to cut the plastic stands down.

Once I got this far it was time to test the Data Logger out. I downloaded the RTClib and SdFat libraries and installed them. With a few minor tweaks to my script for Monitoring my house power with an Arduino and Perl I had the real time clock set and logging dates/times but I've not yet tested the SD card so more to follow.

Comments

data logger shield now in use in my home energy monitor

Having played with the data logger for a day I've now incoroprated it into my home enery monitor. I hit s few issues but they were easily resolved:
  1. The function create_file attempts to create a file named YYYYMMDD.csv. It does this by getting the year and multiplying by 10000 then adding month * 100 then adding the day and finally converting to a string. I kept getting the wrong answer even though the total was stored in an uint32_t. It was the age old problem of uint32_t = uint16_t * 10000 calculating the year * 10000 as a uint16_t then promoting it to a uint32_t (so I got overflow). Easily fixed with a cast.
  2. I did not read the SdFat docs properly and did not realise you have to call sync() to actually write the data to the SD.
Here is my modified code for my energy monitor:
#include <EEPROM.h>
#include <Wire.h>
#include "RTClib.h"
#include <SdFat.h>
#include <SdFatUtil.h>

RTC_DS1307 RTC;
uint8_t today;

Sd2Card card;
SdVolume volume;
SdFile file;
SdFile root;
// filename to store data on SD card - format YYYYMMDD.csv\0
char filename[13];

const int  dataLedPin    =  4;  // LED indicating sensor data is received
const int  logLedPin     =  5;  // LED flashes during a log attemp

const int logInterrupt = 1; // ATmega 168 and 328 - interrupt 0 = pin 2, 1 = pin 3
const int interruptPin = 3;

#define LOOPDELAY 60000

// count of pulses from the electricity meter in LOOPDELAY ms
// a byte can hold 255 pulses a minute and if we hit this we'd be
// consuming a hell of a lot. If LOOPDELAY is more than a minute
// a byte might not be big enough but if we increase the sie of the
// sensor storage type we'd have to protect all reads/writes to it
// from interrupts
volatile byte sensor = 0;  // Counts power pulses in interrupt, 1 pulse = 1 watt
unsigned long total = 0;  // Total power used today
//unsigned long last_reading = 0;

// store error strings in flash to save RAM
#define error(s) error_P(PSTR(s))

void error_P(const char* str) {
    PgmPrint("error: ");
    SerialPrintln_P(str);
    if (card.errorCode()) {
        PgmPrint("SD error: ");
        Serial.print(card.errorCode(), HEX);
        Serial.print(',');
        Serial.println(card.errorData(), HEX);
    }
    while(1);
}

// Write CR LF to a file
void writeCRLF(SdFile& f) {
    f.write((uint8_t*)"\r\n", 2);
}

// Write an unsigned number to a file
void writeNumber(SdFile& f, uint32_t n) {
    uint8_t buf[10];
    uint8_t i = 0;
    do {
        i++;
        buf[sizeof(buf) - i] = n%10 + '0';
        n /= 10;
    } while (n);
    f.write(&buf[sizeof(buf) - i], i);
}

// Write a string to a file
void writeString(SdFile& f, char *str) {
    uint8_t n;
    for (n = 0; str[n]; n++);
    f.write((uint8_t *)str, n);
}

// create the filename we will write to based on the current datetime
// we return a ptr to the null terminated string for the filename
char * create_file(DateTime d)
{
    uint8_t month, day;
    uint16_t year;
    uint32_t all = 0;
    char *name;
    uint8_t i = 5;
 
    // calculate YYYYMMDD
    year = d.year();
    month = d.month();
    day = d.day();
    all = (uint32_t)year * 10000;
    all += (uint32_t)month * 100;
    all += (uint32_t)day;
 
    filename[sizeof(filename) - 1] = '\0';
    filename[sizeof(filename) - 2] = 'v';
    filename[sizeof(filename) - 3] = 's';
    filename[sizeof(filename) - 4] = 'c';
    filename[sizeof(filename) - 5] = '.';
    do {
        i++;
        filename[sizeof(filename) - i] = all%10 + '0';
        all /= 10;
    } while (all);
 
    name = &filename[sizeof(filename) - i];
 
    // create (if necessary) the file and append
    file.open(&root, name, O_CREAT | O_WRITE | O_APPEND);
    if (!file.isOpen()) error ("file.create");
 
    return name;
}

void setup(void)
{
    char *fn; // filename
 
    Serial.begin(57600);
    Wire.begin();
    RTC.begin();

    if (! RTC.isrunning()) {
        Serial.println("RTC is NOT running!");
        // following line sets the RTC to the date & time this sketch was compiled
        //RTC.adjust(DateTime(__DATE__, __TIME__));
    }
    DateTime now = RTC.now();

    today = now.day(); // store todays day
 
    // initialize the SD card
    if (!card.init())
        error("volume.init failed");

    // initialize a FAT volume
    if (!volume.init(&card)) error("volume.init failed");
 
    // open the root directory
    if (!root.openRoot(&volume)) error("openRoot failed");

    fn = create_file(now);
    Serial.print("Writing to: ");
    Serial.println(fn);
   
    pinMode(dataLedPin, OUTPUT);    // LED interrupt indicator initialization
    pinMode(logLedPin, OUTPUT);

    pinMode(interruptPin, INPUT);  
    // enable 20k pullup resistor:  
    digitalWrite(interruptPin, HIGH);  
    attachInterrupt(logInterrupt, interruptHandler, FALLING);
    interrupts(); // now setup, enable interrupts
 
    Serial.println("Start");
}

void loop(void)
{
    // flash the data LED
    digitalWrite(dataLedPin, HIGH);
    delay(50);
    digitalWrite(dataLedPin, LOW);  
 
    // reading/writing volatile data shared with an ISR needs to
    // be done with interrupts disabled unless the data can be read
    // in an atomic operation e.g., a byte
    if( sensor != 0 ) {
        digitalWrite(logLedPin, HIGH);
        Log();
        digitalWrite(logLedPin, LOW);
       //last_reading = sensor;
    } else {
        Serial.println(sensor, DEC);
    }

    // wait a while - interrupts do fire during delay
    delay(LOOPDELAY);
}

void Log()
{
    unsigned long sensor_count;
    uint8_t oldSREG = SREG;   // save interrupt register
    cli();                    // prevent interrupts while accessing the count  
    sensor_count = sensor; //get the count from the interrupt handler
    sensor = 0;           //reset the watts count
    //last_reading = 0;
    SREG = oldSREG;           // restore interrupts

    total += sensor_count; //total watts counter
    Serial.print(sensor_count, DEC);
    Serial.print(',');
    Serial.println(total);

    DateTime now = RTC.now();
    if (now.day() != today) {
        // if day changed close old file and open a new one
        file.close();
        create_file(now);
        total = 0;  // reset total watts used today
        today = now.day(); // reset today
    }
    // write time, watts, watts_total to SD
    writeNumber(file, now.unixtime());
    writeString(file, ",");
    writeNumber(file, sensor_count);
    writeString(file, ",");
    writeNumber(file, total);
    writeCRLF(file);  
    file.sync();
}

// interrupt from interruptPin
// a pulse from the meter LED, just increment sensor count
// one pulse = 1 watt
// NB interrupts are automatically disabled in a ISR
void interruptHandler() {
  sensor += 1;
}

Reduce power used by Arduino

My next task is to try and reduce the power consumed by this set up. Currenctly we are talking 22mA which rises when the SD card is written to. Obviously I can reduce the calls to sync() to one every 10 minutes or so but I need to investigate sleep modes in the Arduino.

power_adc_disable loses a little power

By calling power_adc_disable (I'm not currently using ADCs) it reduces the power consumption a little (around 0.04 mA).

Outstanding issues with the data logger shield

I have a few outstanding issues with the data logger shield:

  1. When should I see the green/red LEDs light? I am writing a small line (30 or so chrs) to a file then calling sync() every minute. Is this sufficient to see the green LED light? I've tested the LEDs and they are soldered in the correct way around but I don't see them light.
  2. What does PgmPrint do? The error_P function in the examples is commented as "store errors in flash to save RAM". What does this mean and how do I access the errors?

How to use the LEDs on the data logger

Just to answer one of my own questions regarding how to use the LEDs on the data logger shield. They are not wired in by default. You need to attach a wire from L1 and L2 to the appropriate Arduino digital pin. See the following picture where the L1 and L2 LEDs are connected to the arduino D3 and D4 pins.

How to use data logger LEDS