Tuesday, 21 November 2017

Concatenating CSVs the easy way

I was recently asked how to 'merge' a few CSV files into one, and if there was a script or tool that could do that.

Lets say you have 3 CSV files, called 1.csv, 2.csv and 3.csv
All you need to do is decide which CSV you want as the 'master' file (1.csv) then delete the header row from the rest.   Then simply do the following:

In Linux, Unix or OSX

cat 2.csv >> 1.csv
cat 3.csv >> 1.csv

In MSDOS, Windows or any other variant

type 2.csv >> 1.csv
type 3.csv >> 1.csv

>> on a command line means "Append to"

cat / type is simply a command to display the file

So cat 2.csv >> 1.csv is saying read 2.csv and append it to 1.csv.

Tuesday, 19 September 2017

Raspberry Pi Zero Smart Energy Meter

The plan for this project is to take my ageing Smart Energy meter that I got given free about 10 years ago, connecting it up to the very low-cost Raspberry Pi via a serial to USB interface, then using some Python scripts to 'webify' it and send the data to the home server to make graphs and stuff.

Not the world's most complex of hardware hacks as it just involves plugging things together but for a first dip back into the world of that kind of thing, I think it's enough.

Project overview:

Objectives: Getting the electricity usage data into a MySQL database on the server so that I can analyse it and build nice graphs and stuff and things.

Challenges: Parsing the stream of XML data, then firing it to the server using some kind of API endpoint (that I have to write)



  • Pi Zero ordered!


  • Pi Zero delivered!

09/05/2016 - 18:20

  • I just successfully booted the Pi Zero, connected to the old USB hub. Not a major milestone, but a very important one.

It didn't explode! Yay! 

August 2016

  • We moved house and the Smart Energy meter has gone missing - it's in a box somewhere

September 2017

  • Still no further progress but I'm posting this project as it stands in an effort to kick me into actually making progress.  Hopefully there will be an update soon! 

Posting old stuff that I never finished - A "Weekly links" from sometime in 2014!

As part of an effort to post stuff that has been in draft for three years - here's this..   A "Weekly links" from sometime in 2014!


"Weekly"  !?!!

Anyway, moving swiftly along before anyone notices..

Some cool stuff I have found recently..

The story of the Amstrad CPC464 via The register

A Fish Tank on Wheels - drivable by the fish. No really.  via Hackaday

Monday, 4 September 2017

New theme

The old colour scheme was hurting my eyes.   Think I must be getting old.

Sunday, 3 September 2017

Raspberry Pi DVR - Streaming TV over the LAN

In our house we have a TV in an area where we don't have an aerial socket.  We hardly ever watch live TV any more anyway as we have an Amazon Firestick -  so it's not been that much of a problem.

Occasionally, however, we want to watch something that's live and no all the channels have 'Watch Live' as an option on their app. (Channel 4, I'm looking at you!)

So, last week I was reading the Pi Hit's blog and saw a short note about a DVR project that can stream live TV over the local LAN.   Just what we needed.

Friday, 21 April 2017

Oops - Time Flies when you're having fun..

Time flies..  or Chronosquittos?
So it's been a little while since the last post...

This, however, is a new start.   I'll leave all the old stuff because it's pretty good reference material still and most of the links still work.

I hope to be doing more stuff to put on here. Obviously I have done some bits in the last four years too so I'll add some of that as I go.

The main technology changes in the last four years, I would guess, are to cost and batteries.  You can get a Raspberry Pi Zero W for £10 which vastly out-powers any £1000 PC that I had 15 years ago, and has WiFi built in, and you can get batteries that recharge in hours that will power a heavy duty hammer drill or lawn mower.  Oh, and 3D printers.  Those are a big thing now.

Tidying up some loose ends from old posts - I can sync my iPod again! I managed to re-incarnate it to a level where it will sync with my work macbook pro, but I've stuck to my word and not tried to update the OS, so it's still running an iOS version from, like, 1400BC or something, but it's perfect for playing bed time stories to our son, so that's cool.

Concatenating CSVs the easy way

I was recently asked how to 'merge' a few CSV files into one, and if there was a script or tool that could do that. Lets say you h...