The Rustlup Team at Hackathon London – June 2012

Last weekend (29 Jun-1 Jul 2012) I attended Mark Pearson‘s Hackathon London. This was the second I have been to – the previous one was back in March.

The basic format of the weekend is arrive on Friday evening and hear pitches of ideas from attendees. From this, teams are formed and they work on the idea from Friday evening to Sunday afternoon. The weekend ends with presentations and demos from the teams.

I went along with Nathan and Mike. We work together on a daily basis and they both have been to the Python and Android hack days held in my flat.

Friday Evening Pitches

The event starts with everyone mingling/networking. This is the bit of the weekend I like the least, but it was good to catch up with people who I’ve met at previous events. I’m much more a fan of “active networking” which is the main purpose of the weekend – i.e. lets work together on this project, then i’ll know if you’re actually as good as you say you are.

We had an introduction from Mark and a few presentations from sponsors of the event. After that anyone who attended could stand up and pitch an idea. Most of the ideas presented were very good. I ended up joining the Rustlup team – the core idea is to build a business around a better way of finding recipes by using the current contents of your fridge. This was pitched by Troy Kennedy.

Seemed like an interesting idea and from a technical perspective there were lots of potential to get something built over the weekend. We ended up with a team of 9 people to work on the project: 2 designers / front end (Jason and Christian), 4 backend programmers (Me, Mike, Nathan and Graham), 3 business (Troy, Amir, Tunmise).

The Rustlup Team (minus Jason).  From left: Mike, Me, Graham, Nathan, Christian, Amir, Tunmise Troy.

Over free Dominos pizza, we spent a few hours thinking about what the product would look like and the user interactions.  The technical work was split into the following:

  • Front End Design
  • Recipe web scraper
  • Database backend (Django)
  • Facebook login
  • Recipe search tool

A few of us left on the Friday to go home, whilst the others cracked on with the project. I went back to stay with my parents in Essex (yes, to the very same town that TOWIE is filmed in) – there were plenty of drunk orange people on the train I tried to ignore as I worked away on my laptop.

Saturday Morning

Arriving back on Saturday morning, some major progress had been made overnight on the design and my arrival woke up Troy and Christian who were asleep slumped over their laptops.

So first thing was to get the server and environment up and running. I have a spare dedicated server that I setup to serve the website and a github repo for our source control.

Scraper

The core of the project relied on being able to get hold of recipes for the content – which we decided to scrape off the web. We discovered that Google indexes these and has encouraged owners to markup their sites using semantic web “Rich Snippets”. This made the task significantly easier and over the entire weekend we managed to get about 10,000 recipies. There were some problems with the data and we had to create an algorithm to extract the clean ingredient name – e.g. “1 (15 oz (450g) ) container mascarpone cheese, at room temperature” becomes just “mascarpone”. Mike and Graham did a great job writing the Python code to parse and get this data.

Lunch

As a team we took a trip out for lunch. Was a lovely sunny day and we headed towards a place we went last time – Thames Take Away, a Chinese on Tanner Street. Being disappointed that it was closed, we went over towards More London Place and the shops over there. Was a nice to break to get out of the office and relax as a team in the sunshine.

Recipe Search

For the core of the idea, there needed to be a search that takes the input ingredients and find recipes with the same ingredients. In the spirit of a hackathon, we decide that we should use Redis to do this task just because we knew it would be fast over lots of recipes – and it’s a piece of cool tech that none of us had used before. Mike did most of this work but unfortunately we ran out of the time to get it properly working and integrated with the site. It works reasonably well for multiple ingredients but “lemon hazelnut tiramisu” seems to appear as the top result far too often.

Bar Code Scanning

Throughout the weekend, we were continually discussing the best way of the getting the user’s ingredient list into the site. Using the site to manually type in items is a little difficult – especially on mobile devices. So we spent some time investigating using a mobile to scan the products.

Late Night Coding

First off we looked at using the bar code. But following a trip to a local Sainsburys, we discovered that for own brand items, the bar code has a custom (presumably internal) number on it. We also looked at the Sainsbury’s  online shopping site, but none of the numbers consistently matched up. After the weekend, I did some more research and found that we could have done it with Tesco own brand as the bar code on their products can be found on their online shopping site.

We also tried using a mobile phone to take a picture and then OCR the image to extract the name of the ingredient. But the online OCR tools didn’t recognise the text well enough. Even if it did work, we’d have a big task to work out actually what the key words were on the packet.

Android App with Pusher

Some time in the early hours of Sunday, Christian and I decided it would be a good idea to investigate using voice recognition to help with the input.

I found a tutorial showing how to use the voice recognition software built into Android, so I created a quick app to do this. From testing it detects words remarkably well, although some food words are problematic – it missed most non-English words and it sometimes detected “pasta” as “pastor”. But we knew it worked good enough for a controlled demo.

After getting the recognised words, we need a way to get this to the website. Since Pusher sponsor the hackathon, we decided to use their real-time messaging service to hook the app to the web browser.  We adding a Javascript library to the main page that then was sent data from the Android app. This then used JSON calls to lookup ingredients in the database. Luckily there were libraries in place for all of what we needed – although the documentation was a little incomplete for Android because it was a 3rd party library. When we got stuck, Phil from Pusher was on hand to help us out – even at 3.30 in the morning!

So we ended up with an Android app where you could say ingredients (limited to one word each) which would then be input into the site. Saying “rustl” (recognised as “russell” by the software) would submit the page and give the search results.

Final Integration

Sunday Morning

After getting some sleep I had the task of integrating all the code we had together.  This was frantic to get all the design changes, the Redis search code and all the crawled recipes onto the main site. Mike was running around like crazy using all our machines to processing the 10k scrapped webpages and getting this into the database.

But we eventually got it done for the 4pm deadline and ready for the demo.

Demo

The weekend ended with everyone congregating around the presentation area and each team giving a 3 minute presentation and 2 minutes of questions. In the end there were about 25 presentations (and we were about number 22).

Rustlup home page

Troy from our team give a short overview of the business side of Rustlup and then Christian and I gave the demo of our website. We showed off the search results and then the voice recognition demo. Of all the teams that presented, we were one of the few teams that showed off a working live website and app.

Prizes

Presentations

After a short break and packing up our stuff, we returned back for prize giving. Rustlup was a double winner!!! 2nd prize (Mentoring and £40 Amazon voucher) and the Pusher prize (which gives us a 6 month startup plan worth $294).

Conclusion

Having attended the previous Hackathon London, I was impressed with the improvements they had made in the running of the event. The Friday evening pitching and the Sunday presentation was much slicker and organised.

As for the Rusltup team, I was impressed with how much we managed to get done in the time, how everyone helped each other out and how much drive, energy & focus there was to get the task done. Plus we had loads of fun joking around and playing with exciting bits of technology.

Forming up a team in such a short space of time that worked so well together, is a testament to the quality of developers that these kind of events attract. I have been in other dev teams that had been together for months that never worked that efficiently.

Many thanks to Mark and his team for organising another brilliant weekend. I’m looking forward to going to the next one – when ever it is.

All my photos from the weekend can be found on here.

2nd Place Certificate

Android Hack Day – Brighton, 23 Jun 2012

Last Saturday (23rd June) was the second hack day I’ve organised. Eight of us (Me, Colin, Dave, Matt, Mike, Naomi, Nathan, Tony) got together in my flat in Brighton for a day to create an Android app.

We all have Android phones but none of us had any experience writing mobile apps for it. So we set ourselves the challenge of creating a game – an online version of shove ha’penny.

The game

The basic premise of the shove ha’penny game is to push a coin along a board so it slides and stops between the lines on the board (called a “bed”). Points are scored if you get a coin in each beds.

So for a mobile game we modified it slightly and came up with the following simplified spec to give a chance to get it working within a day:

  • Coins are launched with a swipe on the screen (speed and length of swipe determine how far the coin goes)
  • Coins move in one direction only (up the screen)
  • Only one coin in play at a time (there is no collision detection)

Development

Following on from what we learnt from the Python hack day I setup a github repository before the day, created a basic outline of the project (loosely based on the Android SDK accelerometer demo) and pestered everyone to ensure they had Eclipse, ADT and the emulator working on their laptop.

I think this made a big difference as everyone was ready to start development when they arrived. It also got everyone excited about what we going to create and thinking about how to use Android.

Starting

Dave coding on the Floor

Everyone arrived around 10ish and we set about dividing out the tasks. Roughly this broke into:

  • Scoring storage, players and high scores
  • Basic physics engine (how the penny moves and stops)
  • Game scoring, turns and players
  • Swipe detection and measurement

We then split into teams of between 1 and 3 people.

Physics Engine

I worked with Dave on the physics engine – which determines how the pennys move on the screen. We decided to use the verlet integration to calculate the position of the penny. This works well because the period between drawing the screen (i.e. calls to the Android view’s onDraw() event) might not be constant, so the verlet integration attempts to correct for this.

We battled with working out how to apply friction to the penny moving. Eventually we settled on the quick solution of applying a negative acceleration to the verlet algorithm, detecting when the penny is about the reverse direction and then preventing it from moving further. A bit hacky, but it does the job.

Git

Like the last hack day and most of my home project development, we used git and github for version control. I have just introduced Mercurial to my team at work, so I wanted to see how git compared in a multi-team environment.

We decided to use branches for each feature so each team had a branch to work in. This worked very well and merging changes together was very easy – any conflicts were encountered were trivial to fix.

Compared with Mercurial, the only thing I liked less in git was that branches are not automatically synced upstream and non-ff branches was a pain as we lost that some commits were part of a feature branch, not master. But the github interface is brilliant and easy to see the changes visualise how branches are merged.

Android

Matt and Dave play the game

The Android SDK is very easy to program for and the tutorials go a long way to explaining how to do the basic stuff. They have done a good job of wrapping up the complexity of the system, and stuff like interacting with the phone hardware (sensors, screen etc) is very easy one you have the right parent classes and listeners registered. And the huge community online makes it very easy to find out the best way of doing something.

As ever, it’s always better to reduce coupling and keep the development framework (i.e. Android in this project) out of as many files as possible. We specially tried to create classes that had no Android imports (they just used the normal Java libraries) so they were easy to test with JUnit or run direct from the command line. Our physics code and storing of game turns and scores were created this way. Doing it this way makes testing easier, development is easier to make concurrent and it’s less likely for development to be slowed waiting for someone else to fix code so you can see yours in operation.

Conclusion

The finished app

By the end of the day we managed to have the game play of the app working, and we finished the day with Dave and Matt having a two player game on a phone (Matt won). We didn’t manage to get the high scores database working on the day, but I managed to fix the main bug with it on the Sunday.

Overall I think everyone had a good time (see all pictures of the day). We had lots of fun, learnt lots about Android and actually got a playable game completed in a day. The next step is to fix a few more bugs, test on more devices, polish the interface and then release it to the world via the Android market. Then we can also think about any more features to add (there are plenty in our heads).

Fedora 17 on the Samsung Series 9 2012 (NP900X3B)

Just a quick update to my previous post about Linux on the Samsung Series 9 2012.  I have now upgraded the laptop to Fedora 17 and most things that were a problem are now working.

I have had this laptop for 3 months now and use Linux on it almost everyday.  I only ever boot into Windows 7 to watch Netflix.  It is an absolutely amazing laptop: fast, brilliant screen, easy to type keyboard and it looks great.

Trackpad

Works out the box in FC17.  No changes required. Right click works perfectly now and so does two finger scroll.

Keyboard backlight

The keyboard backlight now turns on when the laptop is in low light conditions.  However, the function keys don’t work to adjust the brightness.  So if you do want to do that you have to use the command line (must be run as root):

echo N > /sys/devices/platform/samsung/leds/samsung\:\:kbd_backlight/brightness

Suspend

There is a still sometimes a problem with the brightness when resuming from suspend.  I have just got used to it and enter my password to a black screen and alter the brightness.

A fixed Vimeo Downloader for Linux

I came across a video on Vimeo that I wanted to download to my computer so that I could watch it later.  The Vimeo Downloader script for Linux no longer works as Vimeo have updated their site.  So I fixed it:

https://raw.github.com/johnteslade/vimeo-downloader/master/vimeo_downloader.sh

Source code repository: https://github.com/johnteslade/vimeo-downloader

Usage

Run the following from the command line:

./vimeo_downloader.sh http://vimeo.com/1084537

Update 28th Jan 2014

This script was broken following some changes made to the Vimeo site.  The script on github now works again.

However, if you would like a cleaner interface I would recommend using the youtube-dl tool which, despite the name, supports Vimeo downloads. This tool is available in pypi and in many Linux package repositories.

Convert tabs to spaces for Python code

I used to develop Python code with tabs for indentation.  I don’t want to get into the religious debate of tabs vs spaces, but the Python community goes with 4 spaces for indentation.  So to allow ease of importing others code and contributing to other projects I decided to switch to spaces.

Updating Code

Update all your files is easy with find and sed:

find -name "*.py" | xargs -n 1 sed -i 's/\t/    /g'

Note: I would strongly suggest you do this on files that are under version control in case something goes wrong.

Vim Settings

Want to use spaces for Python code, but everything else using spaces?  Then add the following to your .vimrc:

au FileType python set expandtab
au FileType python setl shiftwidth=4 tabstop=4

Linux on the Samsung Series 9 2012 (NP900X3B)

On Wednesday I collected by brand new laptop – the latest Samsung Series 9 (model NP900X3B). I’m a Linux user so this post is about getting Fedora 16  running on it.

Updated: 27 Jun 2012 added new post for running Fedora 17 on this laptop

Updated: 28 Mar 2012 added mouse details and backlight adjustment

Updated: 26 Mar 2012 with more details

If you prefer to run another Linux distribution see these blog posts: Debian by Daniel Barlow and Ubuntu by Vaibhav Verma.

Booting from USB

First step of getting Linux installed is to boot from a USB stick.  This took me ages to get working – in particular you must disable “Fast BIOS Mode” in the BIOS setup (it’s on the “Advanced” tab).  Then you can alter the boot priority to USB first.  Thanks for this thread on Notebook Review for pointing that out.

My Fedora 16 x86_64 live USB stick booted fine and everything seemed to work ok.  So next step was to install it to disk.

Re-partitioning

In order to make space for Linux I shrunk the main partition (originally 90GB in size) down to 50GB using the Windows Disk Management tool.  This left 43GB of unallocated space to install Linux on.  I also had to delete the 5GB hibernation partition using gparted.  This has had no effect on how Windows runs.

Trackpad

The biggest problem I found in Fedora was that trackpad has didn’t work properly.  It worked fine in the original 3.1.0 kernel – but when I updated to 3.2.10 and 3.3.0 it started behaving erratically and the cursor not moving when I moved my finger.  Add the following to your xorg config (thanks to Guy Lunardi for this):

Option "FingerLow" "1" #sensitivity detection low
Option "FingerHigh" "1" #sensitivity detection high
Option "RTCornerButton" "3" #right-click to bottom right
#Option "MinSpeed" "0.7"
#Option "MaxSpeed" "1.7"
Option "TapAndDragGesture" "1" #tap&release then tap&drag
Option "PalmDetect" "1" #avoid bad track behavior
Option "VertTwoFingerScroll" "1" #two-finger vertical scroll
Option "VertEdgeScroll" "0" #right edge vertical scroll
Option "TapButton1" "1" #one-finger tap = left-click
Option "TapButton2" "3" #two-finger tap = right-click1

There are still some problems – you can’t tap the touchpad to click and right click is broken.  But two finger multitouch works to scroll up and down pages.

Backlit Keyboard

The keyboard backlight keys (f9 and f10) don’t work as default.  However, there is a fix for first series 9 laptop (details here) that works on this laptop.

  • Download https://github.com/iksaif/samsung-laptop-dkms
  • Compile the code (you need to have kernel-devel, kernel-headers and gcc installed)
  • Load the module with insmod
  • Change the brightness (N=0 for off, N=8 for max) echo N > /sys/devices/platform/samsung/leds/samsung\:\:kbd_backlight/brightness

There is more work to automatically load this module and map the keys.

Suspend

I initially thought that the laptop would not resume from suspend/sleep.  But on further investigation it is just a screen brightness problem.  When it resumes the brightness is so low the screen looks like it is off.  But you can enter your password, hit enter and then sometimes the brightness comes back – or if not hit Fn+F3 and it works again.  I will get a bug raised for this.

External Monitors

The laptop has two external monitor ports – MicroHDMI and VGA (via a dongle.  Since I don’t have the VGA dongle I could only test microHDMI and it works fine under Linux.  I was able to drive 2048×1152 monitor as a second screen and drag windows between that and the laptop screen.

Wireless

Wireless worked out the box, connecting to my WPA2 home network.  I’ve had no problems with connectivity.

Sound

The sound works out the box (and the speakers sounds great for such a small laptop).  The headphone jack works too but I haven’t tested it with a microphone (it’s a dual headphone/microphone port).

Graphics

Again out the box graphics works perfectly.  Works in 3D acceleration perfectly – I’m running Gnome 3 shell.   glxgears (yes it’s basic) runs but there is some tearing when in fullscreen mode.

Battery Life

I haven’t done any tests on battery life, nor have I changed any settings.  From a full charge, Linux is estimating about 3 hours of time remaining (windows reports about 5).  This is my next item of things to check out – trying out powertop, Jupiter and tuned.

Ethernet

Ethernet is via a breakout dongle (code AA-AE2N12B and supplied in the original box).  This worked perfectly in Linux: plugin dongle, attach ethernet cable and good to go.  The only annoying thing is having to remember to carry around the dongle.

Webcam

Tried this out using Cheese and Skype and both the camera and internal microphone and work fine.

Conclusion

Overall Linux is working very well on this laptop.  There is some more work to do be done, but I am surprised how much worked on the default Linux installation.

If there is anything else anyone would like to me test out let me know in the comments.

I am also updating the LinLap entry for this laptop.

Brighton Python Hack Day – 11 Feb 2012

On Saturday 11th Feb 2012, Nathan Humphreys and myself ran a Python Hack Day for a few people at my flat in Brighton.  This was the first hack day that any of us had run or attended so we didn’t know what to expect.  But we learnt lots of things along the way.

We chose Python as a language to use since plenty of people we knew had a basic knowledge in it – plus Nathan and I have been using it side projects for a while now.

Start of the day

Around 10am on a Saturday, 5 of us are ready to work on a Python project, and the first item is to decide what to work on. No one was really that bothered so we decided on a project that had the most scope – creating a photomosiac web app.  We decide that we would develop this on Google App Engine and split into two teams – one for the web front end and the other for the image processing backend.

The first thing we ended up doing was setup the development environments.  We must have spent the first 2 hours getting python, git, Python Imaging and the App Engine SDK working correctly on all our machines.

Once we started coding everything went smoothly, with plenty of code being written and pushed to our github repo. I had never used git before in a team environment before – only for my own projects where I have 1 developer and 2 branches. I was amazed at how easy it is to merge files and push your changes across the network to someone else.

Afternoon

Following our lunch, we made good progress and was ready to start integrating and testing the algorithm in late afternoon.  Then it was time for intense pair debugging to get this working.  After discovering a few oddities with GAE Image functions, we finally got something basic working by 7pm and continued tweaking it further until 9.

Lessons Learnt

After the first hack day, I learnt quite a few things:

  • A 9 hour day is actually quite short, so maximise the available time.  Next time, we will try and select an idea to work on using email, create a skeleton project and ensure everyone can compile and run the project.  And all this before they arrive on the day.
  • Git and github rocks
  • Even in a single day, 5 people can create code that becomes difficult to debug and test.

What’s Next

Nathan and I have plans for more hack days as they are brilliant fun.  Next up we are going to attend some else’s hackathon – Mark Pearson’s Hackathon London on 23-25th March 2012.

My upgraded development workspace

It was time to upgrade the workspace at home where I do my software development.  It was getting uncomfortable to sit there for many hours at a time and that’s not what you need when trying to code.  So I went off to IKEA.  Here’s my old desk setup – a bog standard dining table with a cheap chair from Rymans.

Chair

As a tall person (6ft 3) I’ve always suffered with chairs and desks being the wrong height for me and suffering neck ache.  So first item up is a new chair that has a high seat and a head rest.  Where I used to work I had an amazing RH Logic 4 chair, but not having a spare £900, I opted for the £125 IKEA MARKUS.  It’s a very comfortable chair with plenty of back support and it reclines too.

Desk

Then to solve the next height related issue of mine: the desk.  So I added four IKEA VIKA KAJ legs that are adjustable between 60-90 cm.  I set them to 80cm which works well for me.  And while I was there I added a IKEA SIGNUM cable tidy to clear up the mess of cables behind the desk.

Monitors

My final update is to my dual monitor setup – a Dell 23″ and a Samsung 20″.  Again, getting the monitors to the correct height is important, and the cheapest option I found before was to use two Ergotron Neo-Flex LCD Stands. But these don’t go very high and they struggle with the weight of the 23″ monitor.  So I ordered an Allcam MDM05 Multi Screen Desk Mount Bracket from Amazon.  Was very impressed with the next day delivery and the couriers (DPD) texted me in the morning to give me an hour slot when the delivery would occur.

The screen mount was very easy to assemble but you have to tighten up all the bolts which makes it difficult to tilt and move the monitors.  This is fine for me as I want them fixed in position.

So here is my dual monitor setup on the stand.  It looks good and leaves the area under them clear.

Complete

So here is the final view with chair, desk and monitors.

What’s Next

I’m not done yet, but this is the first pass.  Having now raised the desk, it is a little unstable and does wobble when I’m typing.  So I need to get a more solid table top – probably an IKEA VIKA AMON.  I might also get some IKEA DIODER lights that change colour to light up behind the monitors.

And then it’s to computer hardware upgrades – a wireless keyboard & mouse and a new laptop.

… As A Service – SAAS, PAAS, IAAS

Thanks to nkhumpreys I was introduced to the three “.. as a service” concepts from cloud computing.  Even though I have been using the cloud I had never heard the terms before but they classified the tools I had been using. Here’s my overview.

SASS – Software As A Service

The top of the stack where the user interacts with an application which handles the processing and storage of data. Like a desktop application you access through a web browser.  The user is not usually developing code.

Examples: Google Docs, Dropbox, SalesForce.com

Typical Billing: on a monthly subscription

PAAS – Platform As A Service

A software platform is provided which abstracts away some of the elements of the cloud – normally the scalability elements. This allows a developer to write a piece of code that processes a job (e.g. output a page of a website) and not worry about how many concurrent instances there are. This is great for services that might have unpredictable demand or where there are bursts of activity – such as algorithm that runs for 5 minutes every hour.

Generally the platform makes restrictions on the type of languages of the software and the code must be written in a way so it can execute in parallel – think problems that would work under Map Reduce.

Examples: Google App Engine, PiCloud

Typical Billing: based on resource usage at a granular level – e.g. amount of bandwidth or number of CPU cycles used.

IAAS – Infrastructure As A Service

The renting of a piece of infrastructure for a short period (e.g. a server) in small increments (often hourly).   This reduces the need to keep spare servers to deal with load fluctuation – new servers can be rented on an hourly basis when required.  This provides full access of the instance server including shell access and installed libraries.

Examples: Amazon EC2Rackspace Cloud

Typical Billing: per instance per hour

Undefined subroutine &Scalar::Util::weaken

Ever had the problem in Perl:
Undefined subroutine &Scalar::Util::weaken called at
This is caused by a version of Scalar::Util that is perl only and doesn’t implement the weaken function (ref).  To install the XS version of Scalar::Util use cpan and run:
force install Scalar::Util