Saturday, December 29, 2012

Software Installation

So we have our RasPi up and running but we need to do a few housekeeping tasks first and install some of the software we need to make everything work.

First of all lets make sure all our software is up to date

Make sure you are logged in and connected to the network. From the command prompt enter the following two commands
sudo apt-get update
sudo apt-get upgrade

The first line update your package repository so it knows about the latest versions of all the packages, the second will the upgrade the packages you have installed.

Now lets secure the system and change the default password.
passwd
Then enter you existing password 'raspberry' and then enter a new password, twice. That's now changed the password for the pi logon

Now lets install the software we need
sudo apt-get install gpsd gpsd-clients
sudo apt-get install python-pygame 
 Next lets check our GPS is working. Take a look at the previous post on setting up GPSD 

You don't need to re-do the install bit again but might need to kill gpsd and restart it to get cgps to work.

Now if we want to shut everything down to power it off
sudo shutdown -h now
Unplug the power from the USB hub and we are shutdown cleanly.

or if we want to down a re-boot
sudo shutdown -r now

Setting Up the Raspberry Pi

So next step.

We have a working python script to grab GPS data

We have a python script to plot that data on the screen.

Now we need to get it going on the RasPi.

So the hardware we will need

1. RasPi Model B
2. Powered USb Hub
3. 4GB SD Card
4. GPS Receiver.
5. Keyboard and Mouse
6. Cables Video and some USB Cables

Step 1 - Put the OS on the SD Card

Instruction are here. I'm using the Debian Wheezy Image

Step 2 - Connect up the Hardware

NOTE - Don't connect the power to the USB Hub until you are ready to power up !!!!

a) Connect Network Cable to your router (We'll WIFI it later)
b) HDMI to a screen
c) Connect the USB Hub (1 connection to the USB Ports, another one to the RasPi Power Connector) 
d) Connect the keyboard and mouse to the USB Hub
c) Connect the GPS to the USB Hub
d) Put the SD Card into the RasPi

You should have something like this.


Step 3 - Startup and Configuration

Now switch on the TV or Monitor and tune to the right channel and then plug in the power to the USB hub.

All being well the RasPi should boot to a setup menu.

Run through the setup, configuring Locale, Keyboard etc.

And Enable SSH !!!!! This will allow you to log into your RasPi remotely with a terminal program like putty from another PC. I find that much easier to do config work from a full sized keyboard\screen rather than using the Media Center Keyboard\TrackPad.

Once that is all done you should end up with a login prompt.

You can then login. The default logon and password is "pi" and "raspberry".

Your RasPi is now up and running. Give yourself a pat on the back.









Friday, December 28, 2012

Plotting Some Coordinates

Right, now we have a few bits working we can now move onto doing something half useful.

What we are going to do today is read from a CSV (Comma Separated Value) file that will hold a list of lat and lon coordinates and them plot them on the screen.

The format of the file is

Header row contains the column titles. So
Lat,Lon
Then each line will have lat and lon coordinates from a GPS. So something like


51.89563,-0.161115
51.894412,-0.160428
51.893975,-0.159377
51.894068,-0.157746
51.894121,-0.156845
51.895511,-0.157424
51.897007,-0.158647
51.897961,-0.15942
51.898437,-0.159956
51.898186,-0.1609
51.897669,-0.161952
51.897021,-0.162274
51.896491,-0.161995
51.896107,-0.161544
51.89563,-0.161115

To create the file I just used google maps and plotted a little course round a couple of roads near me and noted down the lat and lon coordinates of each point back to where it started.

Or if you have your RasPi up and running (or your doing this on a laptop) you could use the logger program in a previous post to generate it by firing it up and driving around for a bit. But as mine is stationary at the moment i manually created something.

ok first bit of code gets the libraries we need and sets some variables

#! /usr/bin/python
# Plotting test using pygame
import pygame, sys, os, csv   # Import the pygame, sys and os libraries
from pygame.locals import * # Get pygame constants
pygame.init() # Initialise pygame
screenx = 640 # Size of window
screeny = 480
offsetx = screenx/2 # middle of window
offsety = screeny/2
firstrow = True
          scale = 40000
Next bit sets up our drawing surface, window title and opens the file to get our data

surface=pygame.display.set_mode((screenx,screeny)) # Create a surface to draw on
pygame.display.set_caption('pygame drawng test')# Set the window caption
screen = pygame.display.get_surface()
gps_file = open('gps_test_data.csv','rt') # Open a file to read data from 
 Next bit is where all the magic happens


try: # Read all the lines in
    reader = csv.DictReader(gps_file)
    for row in reader:
        while firstrow: # if this is the firstrow use coords for center
             firstx = int(float(row['lat'])*scale) 
            firsty = int(float(row['lon'])*scale) 
            firstrow = False
            lastx=offsetx
            lasty=offsety
       else:
            x = int(float(row['lat']) * scale)-firstx + offsetx     # make this point middle of the screen
            y = int(float(row['lon']) * scale)-firsty + offsety
            print firstx,firsty, lastx,lasty,x,y
            pygame.draw.line(surface, (0,0,255),(lastx,lasty),(x,y)) # Draw a line
            pygame.display.flip()
            lastx = x
            lasty = y


So bit of explanation is due
reader = csv.DictReader(gps_file)
for row in reader: 
These lines creates a reader object to get data from our file. It uses the csv.DictReader to put the data into a dictionary object. Have a read up in Python about them. The for part loops through the file and executes the next set of statements on them.

        while firstrow: # if this is the firstrow use coords for center
             firstx = int(float(row['lat'])*scale) 
            firsty = int(float(row['lon'])*scale) 
            firstrow = False
            lastx=offsetx
            lasty=offsety
Right the next bit will only execute on the first set of coordinates because we need to get the start point for our first line. To get usable x and y coords in the window range (640x480) we need to multiply the coordinates by our scale. You might need to adjust the scale depending on how far apart your coords are. We also assume that our first coords will be plotted in the middle of the screen, lastx/y.

 else:
            x = int(float(row['lat']) * scale)-firstx + offsetx     # make this point middle of the screen
            y = int(float(row['lon']) * scale)-firsty + offsety
            pygame.draw.line(surface, (0,0,255),(lastx,lasty),(x,y)) # Draw a line
            pygame.display.flip()
            lastx = x
            lasty = y
So, here's what we do for the rest of the lines. We calculate the next x and y coordinates, draw a line from our last coordinates to the new coordinates, display the line on the screen and then we set lastx\y to remember our new last coordinates. And we keep looping round until all the coordinates are plotted. Simples.


Finally:
    gps_file.close()
def input(events): # Function to process pygame events
    for event in events:
        if event.type == QUIT: # If we click ALT+F4 programme will quit
            sys.exit(0)
     
while True: # Keep looping getting events
    input(pygame.event.get())
This last part of the script closes the file when all the coordinates are plotted. Also has the function for the event handler which will loop until we get a quit and then closes the program down.

The full source code can be found on the source code pages.

So if all is working, and you used my test data you should get something like this



 
     



Thursday, December 27, 2012

PyGame Intro

Before we get into plotting the GPS information we need to have a way to do it. Python doesn't come with any inbuilt graphics but there are a number of additional libraries that provide the functionality.

After a bit of research it seemed PyGame is one of the more popular ones to use. After a quick read on their website and tried out a few of the tutorials it looks pretty easy to do what I need it to.

As the name suggests it aimed at games programming, so should easily be able to hand a little line drawing.

Have a look at the tutorials and the cheat sheet is really useful.

In it's simplest form to draw with PyGame you

1. Import the PyGame library
2. Initialise it.
3. Declare a surface. i.e. Something to draw on.
4. Draw on your surface
5. Tell PyGame to show what you have drawn.

You also need event handler which basically loops and processes events like key presses, mouse movements, instructions to quit.

So you just setup your event handle to wait for a quit event then close everything down gracefully.

A simple program to do this would be


import pygame, sys
from pygame.locals import *

# set up pygame
pygame.init()

# set up the window
windowSurface = pygame.display.set_mode((640, 480), 0, 32)

# set up the colors
BLUE = (0, 0, 255)

# draw a blue line onto the surface
pygame.draw.line(windowSurface, BLUE, (60, 60), (120, 60), 4)

# draw the window onto the screen
pygame.display.update()

# run the game loop
while True:
    for event in pygame.event.get():
        if event.type == QUIT:
            pygame.quit()
            sys.exit()


Next post we'll put some of this into practice. We'll use some GPS Lat and Lon coordinates in a file and plot them on the screen.

Prototyping in Python

Before I get into 'proper' coding I like to prototype what I'm trying to do so I have a better understanding of how everything works and if there are any gotcha's. So the code in the next few posts isn't fully formed, just hacked together to test the basic workings. I can then re-use it later as I start to build the full program.

For this project I'm going to use the Python programming language which should already be pre-installed on both the RasPi and your Debian Wheezy Virtual Machine.

I've never coded in Python before, so should be fun learning the language. I've coded in most of the modern languages and if you've had some coding experience then Python should be pretty easy to pick-up.

First a bit about Python.

Python is an interpreted language, and not a compiled one. What that means is that it 'interprets' one line of code at a time and then executes it, then moves onto the next. A compiled language takes all the code, converts it to and executable programme and then runs the compiled program.

The advantage of an interpreted language is mainly speed and simplicity to write the code, you just type the commands and the 'interpreter' executes them right in front of you. Where a compiled language can be a pain to get the code compiled to run, but when it's compiled it will run quicker as it's been converted to a faster machine language.

I don't intend teaching you how to use Python, there are good tutorials on the Pyhton website here. Have a look under documentation and tutorial. If you have some coding experience already have a look through the first 3-4 chapters of the tutorial and then i'm sure you can work the rest out from there when you need to.

Ok, our first little bit of coding to get some GPS data.

First, make sure your GPS and GPSD are working as in the previous post.

We're going to use a Python script, rather than type each line into the interpreter, so fire up a text editor, like gedit and start coding.

Right the script we are going to write uses threading. So we will have a thread running that gets the GPS data in the background, then our main program will then get the data from the thread and output it to a file. This is based on a script created by Dan Mandle and you can get the original at http://dan.mandle.me.

So first line we want is
#! /usr/bin/python
This tells the script where to find the python interpreter. But if you're used to Linux scripting you know that bit :-)


The first few lines import the external libraries we want to use. the gps library is the one that gives us access to gpsd.

import os
from gps import * 
from time import * 
import time 
import threading

We then setup some variables we need and clear the screen

gpsd = None #seting the global variable os.system('clear') #clear the terminal (optional)
We now create the class thread that is going to get the GPS data from gpsd and put it in a variable called, surprise surprise gpsd


class GpsPoller(threading.Thread):   
    def __init__(self):       
            threading.Thread.__init__(self)       
            global gpsd #bring it in scope       
            gpsd = gps(mode=WATCH_ENABLE) #starting the stream of info       
            self.current_value = None       
            self.running = True #setting the thread running to true    
     def run(self):       
           global gpsd       
          while self.running:           
          gpsd.next() #this will continue to loop and grab EACH set of gpsd info to clear the buffer

 ok, the next bit of code is executed first when we run the script. It creates and starts the thread, opens a file for us to write to, then waits until we get a line of GPS data that is Mode 3. Mode 3 means has a good lock on the satellites. It the builds a comma seperated string of data from the gspd data and writes it to the file. Waits 1 second, then loops back round and gets the next line and writes that etc etc etc.

The 'except' bit waits for you to do a control+c to kill the program and then tidies things up. Shuts down the thread and closes the file etc.

if __name__ == '__main__':
    gpsp = GpsPoller() # create the thread
    try:
        gpsp.start() # start it up
        gps_file = open('gps_data.csv','w') # Open a file to write the data to
        while True:
            if str(gpsd.fix.mode) == str(3): # If we have Mode 3, good lock, then write the data out
                gps_data = str(gpsd.fix.latitude) + ',' + str(gpsd.fix.longitude) + '\n' # just print lat and lon data
                print gps_data
                gps_file.write (gps_data)
            time.sleep(1) #set to whatever
    except (KeyboardInterrupt, SystemExit): #when you press ctrl+c
        print "\nKilling Thread..."
        gpsp.running = False
        gpsp.join() # wait for the thread to finish what it's doing
        gps_file.close()
    print "Done.\nExiting."


 Ok, so type that all in, or copy and paste it, and save the file as something with the extension '.py'.

For us to be able to run it from the command line we need to make it executable. So start up a root terminal go to the directory where you saved it, probably your home directory.

At the prompt type
sudo chmod 755 filename.py
Where filename.py is the name of your file.

Now to run it simply type
./filename.py
And if all is working then it should be printing rows of latitude and longitude data on the screen, the numbers will probably be all the same unless you happen to be moving. Let it run for about 30 seconds.

To stop the program hit CTRL+c and you should have a file in the directory called 'gps_data.csv' with the GPS data in. To check it type
cat gps_data.csv | more
So, now we have a way of getting the GPS data into a program and doing something with it.

Next time we'll do something more interesting with it like plot it on the screen.

The full copy of the python code can be found here.

GPSD

Ok now we get to some of the fun stuff. Getting info from the GPS in Linux.

As I mentioned in the previous post the GPS will transmit information on a USB port to be picked up by 'something' that can then take the data, translate it, and do something with it.

You could just use a program to read direct from the USB port and then translate the data yourself.

OR

You could use something to handle all the complicated stuff and just give you the data that you can easily get to from your program.

And that something is GPSD (GPS Daemon). It's a Linux daemon that runs in the background, finds your GPS, does all the configuration stuff, then transmits the formatted data over a TCPIP port (2947).

And it comes with a pre-build library for a number of programming languages so you can just call functions from your program to get the data you need.

The GPSD website can be found here so you can read up a bit more about what it does and how it works.

First things first, we need to install it onto our VBOX, we'll need to do this on the RasPi when we get it.

So, fire up your Debian wheezy Virtual Machine and login. Start up a root terminal.

To install what we need at the prompt enter
sudo apt-get install gpsd gpsd-clients
 The just follow the prompts.

This will install the gpsd daemon and some gpsd client programs as well, useful for testing and diagnosing any problems. It should also install the python-gpsd library we need to.

All being well gpsd should now be running and connected to your GPS.

You can check if gpsd is running by typing

ps -ef | grep gpsd
That should list a line something like

gpsd -F /var/run/gpsd.sock
If that's the case then gpsd is up and running. If not then re-boot the virtual machine and check again. If it's still not started then run through the troubleshooting guide on the gpsd website.

Assuming gpsd is now running lets see if it can see the gps ok. This is where the gps clients that we installed come in handy.

In the root terminal type
cgps
This will start a programme that will connect to the gps and start displaying information on screen that has been fed back from the gps. See below.



If your getting data through then this confirms gpsd is up and running, connected to your GPS and sending out data ok.

Again, if not then refer back to the gpsd troubleshooting guide.

NOTE:

I have had a few occasions when gpsd has stopped working. It can usually be fixed by either disconnecting the GPS 'prolific' device from the Virtual Machine (right click on USB Icon and deselect the prolific device), then add it back again.

Failing that kill the gpsd daemon and restart it again
sudo killall gpsd
sudo gpsd -F /var/run/gpsd.sock /dev/ttyUSB0
Assuming your GPS is connected to ttyUSB0

NOTE 2:

Although we haven't setup the GPS on the RasPi yet, and it's exactly the same process as above, there seems to be a known bug where the gpsd daemon doesn't always pick up the GPS automatically at boot. You can just do a kill and restart as above to get it going again.

So now we have a working GPS that can be accessed via gpsd. Next step (and post), lets put some code together in python to grab the data and do something with it.

Some GPS Setup

The USB GPS receiver came before the RasPi so I got it up and running on my windows machine first and tested it with the windows utility provided to make sure I was getting data.

Yep, all ok. Good start.

I then configured the VirtualPC to be able to access that USB Port. Fire up your virtual machine and look at the bottom of the VirtualBox Screen (see Image Below) you will see a little USB Icon, right click and select the 'Proliffic Technology ......' line.



That now gives access to the GPS Receiver from the Virtual Machine.

Bit of useful info. A GPS receiver collects a lot of useful info like latitude, longitude, speed and then transmits it to the PC as text over the USB connector. To be able to use that data you need to understand the format of the data it transmits and what 'connection' to listen in on to get the information you need. In the olden days they used to use an old RS-232 serial port, in modern times we use USB.

Ok that's the simple stuff out the way.

In the next post I'll show you how to get at that GPS data on your Debian Virtual Machine.

Test Platform

So while I was waiting for all the bits to arrive I decided to setup my software development platform, it's a lot easier coding on a decent PC rather than on the RasPi.

I don't fancy running a dual boot system between Linux and Windows 7 (my normal platform), so the obvious choice is to run Linux in a virtual machine using some free software like VBOX.

If you're not familiar with VBOX of virtual machines all it really does is allows you to run other operating systems in their own little 'box' or window onto of your current operating system as if it was a PC in it's own right.

Here's a screenshot of VBOX running on my rig.











You'll notice a text editor with some coding going on. We'll get to that a bit later. But to whet your appetite that's some python (programming language) code I'm using to plot GPS coordinates from a file onto the screen.

The VBox software is basically a PC Emulator. You just fire up a new 'Virtual PC' then install whatever operating system you want on it, whatever software you need.

The advantages are you can run multiple OS's on one machine and everything you do in that Virtual PC is isolated from your normal PC, so it's great for trying things out. If you make a mess of it then your normal PC is protected and you can only screw up the virtual PC.

You can configure the Virtual Machines to use your network connection, share peripherals etc, like keyboards, mice, USB devices. And you can configure then so they can access your hard disc so you could share files between you main PC and you Virtual PC.

So I've now setup a couple of virtual machines in VBOX using Debian Wheezy, the version the RasPi uses. The first is my main dev & test platform, the second is a clone of the first after I set it up. Vbox can take a copy of a virtual machine. Saves time re-installing everything "when" you screw something up, you just re-copy the clone and off you go again.

The Platform

First of all I need to decide on the platform to build this thing on. As you can tell from the title that's already decided.

I'm going to base it on the new RaspberryPi mini computer. Theses things are great. Basically a computer on a credit card size circuit board with enough connections to actually be useful. Lots of  people are starting to experiment with these, so loads of community support. And they are cheap. (£24).

Have a look here for more info about this great little unit. RPi Hub.

Basically, they use a mobile phone chipset. They have a couple of USB ports, HDMI, Ethernet and sound output. They have an SD card port that you use as a hard disk to run your operating system.

You have a few operating systems to choose from, Windows is not one of them !!!!! I went for Debian which is a Linux distribution.

I've messed around with Linux in the past so know my way around it enough to get me going.

To get the RasPi going I'll need a few other odds and ends.


  • USB Power adapter, for, well, power really.
  • SD Card for the operating system
  • HDMI Cable, I'm sure I've got one lying around somewhere.
  • Network Cable, yeah I've got them coming out of my ears.
  • Monitor, I'll just use my TV in my office for now.
  • Keyboard and Mouse, I've got a small media centre keyboard and mouse from Maplin - here
  • Power USB Hub. - Got an old belkin one lying around I can use.


Other things I'll probably add later

  • External hard drive or USB Stick for more data storage
  • WiFi Adapter - I have a Netgear N150 I don't use any more that will be perfect.


And finally to get the project going




So that's pretty much all the hardware gathered together for now.

Check the "Spendomiter" to see how much this has cost so far



What's it all about ?

So what is the RaspberryPi Track Day Data Logger ?

Well, I decided to try and combine some of my interests into a little project.

So I'm into my cars and I'm into computers and I love to make things so I came up with the idea of making a data logger for track days.

For those that don't know what a data logger is it's a device which will record all sorts of useful data about your laps as you whiz round a circuit. Kind of like the stuff they have on F1 cars.

So records things like location, speed, braking, acceleration, lap times etc etc. Even video !!!

Commercial units cost anything from £500 to over £1000.

So I think I might be able to knock something together from readily available components, plus with my computer programming background I can write the software I need to run it all how I want it to work.