Saturday, December 1, 2012

HTML5 Geolocation API for thin clients and PhoneGap/Cordova mobile clients

The HTML5 version of the "South March Highlands Trails" GPS tracking application is running as a prototype at the following location:
The application allows anyone with a GPS enabled phone/laptop/device to see their location overlaid on top of the map created by the Ottawa Mountain Bike Association.

Everything GPS is possible via the HTML5 call:

This works well in the iPhone app for "South March Highlands Trails" I am developing that will require a lot of additional work before I can submit it for review at Apple.

1) An application server - WebLogic 12.1.1 (12c) in this case
2) An EAR - An Eclipse managed OEPE EAR/EJB/WAR project set
3) A database - The Derby 10.8 instance shipped with JDK 7
4) Web content (html 5 pages, css, javascript - possibly JQuery)
5) Several installations of PhoneGap 2.5 (for iOS6.1.1, BB10 and Android 4.2 at least)


Wednesday, November 21, 2012

Java 7 SE Raspberry PI Parallel Processing ARM Cluster of 32 boards

Raspberry PI cluster of 32 boards - Under Construction:

Disclaimer: For serious computational power - build your own i7-5820 and use CUDA on an nVidia GTX-970.  The raspberry PI cluster is more of a "build it - and they will come" exploration exercise.

This article is an ongoing discussion of how to get a cluster of 32 raspberry pi boards up and running.

32 node/board Raspberry PI cluster for parallel processing experimentation

The ARM based Raspberry PI board is an excellent platform to investigate various parallel processing configurations.  If we are looking for pure performance then I would stick with an Intel core i7 and a CUDA based NVidia GPU because a single core raspberry PI is about 40 times slower than a single core of an 2nd gen i7-2600 or 3rd gen i7-3610 (about 140 times slower than an 8-thread ForkJoin implementation).  This however is not our goal, we need an efficient and accessible way to run multiple servers - and the pi does this at about $70 per server (board + connectors + 16GB SD) and 4 watts/node.  For example: it would take $9800 of raspberry PI boards with 70G ram to equal one 3rd gen i7 at $1300 with 24G ram.  But we can build a cluster of 8 raspberry PI servers with $560 as opposed to 8 i7 boxes with $10000.

 We need the proper power supplies and network switches for a cluster of 32 raspberry pi boards.
 Raspberry pi boards mount very nicely on standard breadboards using properly bent arduiono headers.

Raspberry PI Cluster
8 board Raspberry PI cluster
In this configuration I am running a research cluster of 8 raspberry PI boards to run distributed Java EE RMI/EJB remote session bean clients of a central Oracle WebLogic 12c server (running on an i7 host)

This tutorial details how to get a networked cluster (bramble) of (eight for now) Raspberry PI boards running as a single distributed auxillary processing unit to a controlling Java EE server - ideally using Hadoop.  The primary goal of this exercise is for distributed experimentation.  As I configure and acquire multiple raspberry pi boards and work out power distribution issues my cluster will increase in size.  I currently work with 8 boards and 8 spares.  The cluster of raspberry PI s can be distributed work using a custom RPC API like remote stateless sessions beans on top of RMI or they can use a formal MapReduce implementation like Hadoop or even MPI.

After running the Oracle embedded ARM JVM with no problems on the REV A raspberry PI using the distribution from Element14, I was not immediately successfull running the JVM on the new REV B (512Mb) version because the default Debian distribution from Element 14 no longer uses the soft float version.  I get the following missing library error.

pi@raspberrypi ~/java/ejre1.7.0_06 $ java -version
java: error while loading shared libraries: cannot open shared object file: No such file or directory

Download a new OS compatible with the Java 7 JDK here

Choose "Soft-float Debian “wheezy”" =

Write it to your SD card

Reinstall Java (curently 1.7.0_10 from Oracle)

You are good to go [Java 7 SE on the Raspberry PI Rev B].  So we can now use Fork-Join, JAXB and JAX-WS webservices (using a single thread however).

DI 1: Powering your Raspberry PI cluster

On some routers you will not get a DHCP assigned address if all the clustered raspberry PI boards are powered up in sync - you will need to stagger the powerup - this only occurs if not enough amps are available.

Using a good Agilent power supply we use from 3 (idle) to 3.5A (startup) (6-7W @ 5V) for 8 boards.

You can use a powered USB router for 4 boards, but 8 will require a better power supply like a bench one from Agilent. A bench supply will usually supply up to 40A power  - but normally 5A which is good for up to 12 Raspberry PI's running at 100% CPU but we will need a better supply for a cluster of 32 raspberry pi boards for example.
A good ATX power supply will suffice to power a cluster of raspberry pi boards.  In this example I have a 450W supply which supplies 30A of 5V power (make sure you put some load on the 5V and 12V rails as well).

Get the ATX adapter and breakout board from SparkFun and make sure you use multiple 24 guage or higher wires to distribute the load (1 wire will overheat, 2 wires go to 28 deg C. - use at least 4 if you go over 8 boards).

As you can see I have yet to fully integrate the power supply interface between the ATX supply and the breadboard bus for the 8 pi's - but we are functioning fine and are no longer limited by the bench supplies or individual 5V USB connectors.  (the blue LED boards are Parallax Propeller 8-core microcontrollers uses a per/core output indicator for now.
I need some sort of protection fuse - in case of a short circuit.  It was very stressful connecting up my 8 raspberry pi boards up to the ATX after testing on one.  I recommend working with all GPIO pins (one 1 header is populated instead of 2 on the latest rev B board) covered by a flat cable connector.

20130126:  I now have 24 of 32 raspberry pi boards powered up however running the full peak 15A off an ATX power supply is not practical as it requires some serious wire guage as my 3 wire 24 guage setup is overheating.  Also if you accidently short the power supply you will use the full 15-40A and burn your wire.  I accidentally shorted the leads on a 5A supply on my metal breadboard and the supply wire started to smell and melt.  This brings us to the recommend way to power a large cluster of raspberry pi boards - separate bench power supplies.  When I shorted the supply the bench supply held stead at 5.2A which is safe enough not to burn your house down before you notice it.


Recommended power supply setup for 32 raspberry pi board cluster

No more than 8 raspberry pi boards per 5A power supply will allow you to add some peripherals like an adafruit display or a propeller 8-core coprocessor on an SPI bus.
So this is kind of expensive but instead of using a 40A bench supply at around $350 I use 4 separate 5A bench supplies (3 Circuit-Test PSC-520 supplies @ 3 x $225 and 1 Agilent U8002A supply @ $450).

DI 2: Updating your board for 512mb RAM (470Mb from 224Mb)

The Rev 2 board has double the ram but will require updated firmware to enable it.
sudo wget -O /usr/bin/rpi-update && sudo chmod +x /usr/bin/rpi-update
sudo apt-get install git-core
sudo rpi-update
- reboot after firmware update

DI 3: Overclocking

The lan chip heats up to 52 degrees celsius from a normal 45 when the raspberry pi is overclocked from 700 to 800 MHz.

DI 4: Setup Networking

Wireless is kind of unreliable, I recommend wired.
The WiPi module from Element 14 works essentially out of the box

Wired networking

After duplicating all the 32 SD cards, put one at a time into one of the raspberry pi boards and change the hostname, hosts and static network interfaces settings

sudo nano /etc/hostname

sudo nano /etc/hosts

sudo nano /etc/network/interfaces
iface eth0 inet static

# here we do not rely on our internet providers's DNS servers - we use the google server at as more reliable DNS server

sudo nano /etc/resolv.conf

DI 5: Setup Java

Setup Tomcat
login in to the manager app using "system:raspberry"

Setup Fortran and MPICH

Issue is that I lower performance (likely network overhead) when I increase the number of nodes (currently 6 pi's)

pi@rpi0 ~ $ mpiexec -f machinefile -n 1 ~/mpich_build/examples/cpi
Process 0 of 1 is on rpi0
pi is approximately 3.1415926544231341, Error is 0.0000000008333410
wall clock time = 0.017286

pi@rpi0 ~ $ mpiexec -f machinefile -n 2 ~/mpich_build/examples/cpi
Process 0 of 2 is on rpi0
Process 1 of 2 is on rpi1
pi is approximately 3.1415926544231318, Error is 0.0000000008333387
wall clock time = 0.020435

pi@rpi0 ~ $ mpiexec -f machinefile -n 4 ~/mpich_build/examples/cpi
Process 1 of 4 is on rpi1
Process 0 of 4 is on rpi0
Process 2 of 4 is on rpi2
Process 3 of 4 is on rpi3
pi is approximately 3.1415926544231239, Error is 0.0000000008333307
wall clock time = 0.037727

pi@rpi0 ~ $ mpiexec -f machinefile -n 6 ~/mpich_build/examples/cpi
Process 2 of 6 is on rpi0
Process 1 of 6 is on rpi1
Process 0 of 6 is on rpi2
Process 3 of 6 is on rpi3
Process 4 of 6 is on rpi4
Process 5 of 6 is on rpi5
pi is approximately 3.1415926544231239, Error is 0.0000000008333307
wall clock time = 0.043331

20121121: Setup 4 networked PIs
20130127: power up of 24 raspberry pi boards


32 Raspberry PI boards from Element 14 @ $35 = $1120
32 Sandisk Ultra 16GB SD cards @ $10-18 = $320-576
0 micro USB cables = $0
1 HDMI cable from Apple = $20
8 power supply cables @ 10 = $80
4 bench 5A power supplies from Agilent or Circuit-Test @224-450 = $896-1800
4 large breadboards (that fit 8 raspberry pi boards) @ 45 = $180
64 bendable arduino headers from @ $1 = $64
5 Gigabit 8 node network hubs or 2 16 node hubs from Dlink @ 65 = $325
32 belkin flexible network cables from the Apple store @ 15 = $480

Total = $3885.00

Copies of this article


Monday, October 15, 2012

Java SE 7 Embedded on Raspberry PI

Getting Java SE 7 Embedded running on the Raspberry PI Linux device

I like the British - they gave cracked the Enigma, they created the Transputer and then the XCore and now they give us easily accessible embedded Java.

Eben - Thank you for developing this excellent device.  Now we can use our corporate development skills in Java SE/EE when we program hardware/software combinations off hours.  The XMOS 4-core/32-thread and Parallax 8-core/8-thread microcontrollers may collect dust now - very unlikely as they don't have an issue with symmetric parallel processing.

It is official - the latest Oracle/SUN Java VM works out of the box for SE Java applications (no ME) on the default Debian distribution for Raspberry PI (Rev B) because of it's use of ARM.

root@raspberrypi:/home/pi/java/ejre1.7.0_06# ./bin/java -version
java version "1.7.0_06"
Java(TM) SE Embedded Runtime Environment (build 1.7.0_06-b24, headless)
Java HotSpot(TM) Embedded Client VM (build 23.2-b09, mixed mode)

I bypassed the configuration steps on the Oracle article and WinSCP'd the GZ directly to the Element 14 provided Debian OS on the Raspberry PI device, extracted it, downloaded a test .class file and was running about 32 times slower than a Core i7-3610M or about 12% the speed of the slowest 1.3Ghz mobile P4 - about the speed of a Pentium III or II.  This slowdown of 2-3 times expected speed is likely due to the JIT or lack thereoff.

The instructions at Oracle are incomplete - you will need to perform extra config detailed at

After expanding using...

tar xvzf ejre1.7.0*.gz

you would get the following if installed to the latest hard-float debian image on the REV B distributed by Element 14 - change to the
pi@raspberrypi ~/java/ejre1.7.0_06 $ ./bin/java -version"Wheezy Armel" soft float distribution.

./bin/java: error while loading shared libraries: cannot open shared

If you didn't do
pi@raspberrypi ~/java/ejre1.7.0_06 $ sudo mv -v ~/java/ejre1.7.0_06 /opt/java
`/home/pi/java/ejre1.7.0_06' -> `/opt/java'
pi@raspberrypi ~/java/ejre1.7.0_06 $ sudo update-alternatives --install "/usr/bin/java" "java" "/opt/java/bin/java" 1
update-alternatives: using /opt/java/bin/java to provide /usr/bin/java (java) in auto mode.

I don't know how I was able to get 2 boards in late Sept 2012 - I ordered them off Element 14 Canada - they arrived in 24 hours - My next 12 ordered on Oct 23 arrived as they predicted on 28 Nov 2012.  A 5 amp bench supply should be able to at least run 8 of them at a minimum power draw of 0.39A/board (with only ethernet connected).  After running 5 headless - I am running at 1.6A.

Java 7 Fork Join on the Raspberry PI

    I took a collatz ForkJoin application and varied the fork from 2^16 to 1 fork.  Results were expected - the overhead of using the single thread in the ForkJoinPool slows down the application.  So ForkJoin should only be used on the controlling server and not on the actual devices themselves.
   Running the ARM at 100% CPU for 2 days caused the Broadcom chip to reach 48 C.
   The speed of the run was around 11 times slower than a P4-3.2 GHz system or 138 times slower than a Core i7-3610.

    So we get about 1.3 MIPS/GHz on the ARM vs 35 MIPS/GHz on the 3610 series i7 without using SSE or GPU processing on either system.

root@raspberrypi:/home/pi/java# ./ejre1.7.0_06/bin/java org.obrienscience.collatz.server.ForkJoinCollatzServer
4446843 ms for: 16
4434227 ms for: 32
4382020 ms for: 64
4342838 ms for: 128
4344761 ms for: 256
4345003 ms for: 512
4315240 ms for: 1024
4306938 ms for: 2048
4287915 ms for: 4096
4263907 ms for: 8192
4360612 ms for: 16384
4751604 ms for: 32768
4467843 ms for: 65536
4386337 ms for: 131072
4340696 ms for: 262144
4273932 ms for: 524288
4301385 ms for: 1048576
4240190 ms for: 2097152
4230078 ms for: 4194304
4237989 ms for: 8388608
4223808 ms for: 16777216

Compare this to a P4-3.2 GHz system with 1-cores/2-threads
388031 ms for : 16
384641 ms for : 32
385609 ms for : 64
385360 ms for : 128
388953 ms for : 256
386562 ms for : 512
383766 ms for : 1024
384453 ms for : 2048
385344 ms for : 4096
382578 ms for : 8192
383656 ms for : 16384
387266 ms for : 32768
382843 ms for : 65536
381438 ms for : 131072
375312 ms for : 262144
374750 ms for : 524288
378547 ms for : 1048576
385407 ms for : 2097152
390078 ms for : 4194304
393062 ms for : 8388608
389297 ms for : 16777216

Compare this to a Core i7-3610 2.7 Ghz system (Asus G75VW) with 4-cores/8-threads - which is around 8% faster than a Core i7-2600 at 2.7 Ghz
32184 ms for : 128
31779 ms for : 256
31322 ms for : 512
31541 ms for : 1024
31079 ms for : 2048
31264 ms for : 4096
30911 ms for : 8192
31186 ms for : 16384
30786 ms for : 32768
31915 ms for : 65536
30998 ms for : 131072
31121 ms for : 262144
31376 ms for : 524288
31779 ms for : 1048576
32347 ms for : 2097152
37294 ms for : 4194304
31644 ms for : 8388608
39499 ms for : 16777216

Next: Running a JPA 2.0 application using EclipseLink 2.4 against Derby 10.8 on Spring in either Jetty or Tomcat on the Raspberry PI 512Mb rev B.

This is our current 8 node raspberry pi cluster - ready for 16 nodes at it has 30 amps (P = VA = (5V * 30A) = 150 watts) of available power.

Eight Raspberry PI boards running as a cluster powered by an ATX power supply

Hinkmond Wong's wiki about low level file based GPIO - nice.

Monday, April 2, 2012

Running both EclipseLink and Hibernate JPA 2.0 Implementations

This article details differences in running EclipseLink (2.3) and Hibernate (3.6.8/4.00) primarily using the JPA 2.0 API and native ORM where appropriate.

persist vs merge:

EclipseLink is 30% faster for inserts using persist than Hibernate is for merge.  This may be the fact that the entities continue to be managed in EclipseLink but are persisted as clones by Hibernate.

Hibernate 5450 insertions @ 14.6 tx-inserts/sec to sybase
EclipseLink 4911 insertions @ 18.9 tx-inserts/sec to sybase

Switch persistence.xml provider tags
public class Drug implements Serializable {
    private static final long serialVersionUID = 3132063814489287035L;
    // keep the sequence column name under 30 chars to avoid an ORA-00972   
    @SequenceGenerator(name="JPA_SEQUENCE_DRUG", sequenceName="JPA_DRUG_SEQ", allocationSize=15)
    private Long id;

    private int version;
    @Column(name="compID")//, updatable=false, insertable=false);
    private String compositeID;

    @Column(name="DIN")//, updatable=false, insertable=false)
    private long din;

EclipseLink Logs:
[EL Fine]: 2012-04-02 11:41:34.596--ClientSession(9012544)--Connection(20436132)--Thread(Thread[main,5,main])--INSERT INTO VIG_DRUG (compID, DIN, DRUG_VERSION) VALUES (?, ?, ?)
    bind => [ORTAB, 0, 1]

Hibernate Logs:

Hibernate: select drug0_.DRUG_ID as DRUG1_0_0_, drug0_.compID as compID0_0_, drug0_.DIN as DIN0_0_, drug0_.DRUG_VERSION as DRUG4_0_0_ from VIG_DRUG drug0_ where drug0_.DRUG_ID=?
insert into VIG_DRUG (compID, DIN, DRUG_VERSION) values (?, ?, ?)

Tuesday, February 14, 2012

Web Services using JAX-RPC, JAX-WS or JAX-RS

Quickstart WebServices Tutorial


Weblogic 12c version now ships by default with JAX-RS 1.1 as a core library.  Previously in Weblogic the following shared web applications were required - now we just use the specification library.

 Library jackson-core-asl(1.0,1.1.1) Active  Library
 Library jersey-bundle(1.1.1, Active  Library
 Library jettison(1.1,1.1) Active  Library
 Library jsr311-api(1.1.1,1.1.1) Active  Library

Now in eclipse just add the JAX-RS facet which will add the jersey servlet to your web.xml

    <description>JAX-RS Tools Generated - Do not modify</description>
    <servlet-name>JAX-RS Servlet</servlet-name>
    <servlet-name>JAX-RS Servlet</servlet-name>


Jax-RPC has fallen out of favor for most new web services in favor of JAX-RS 1.1 or REST.
A simple java SE example that is single threaded and runs off of Metro included in Java SE 1.6.
This example does not need a web container or a full application server to run - however it is non-scalable and thread-blocking simple example.
package org.dataparallel.example.jaxws;

import javax.jws.WebMethod;
import javax.jws.WebService;
import javax.jws.soap.SOAPBinding;
import javax.jws.soap.SOAPBinding.Style;

* definitions targetNamespace="" name="ServiceEndpointImplService

@SOAPBinding(style = Style.RPC)
public interface ServiceEndpoint {
    public static final String URL =;
    public static final String PORT = "8980";
    public static final String SERVICE_NAME = "service";
    public static final String SERVICE_URI = URL + ":" + PORT + "/" + SERVICE_NAME;
    public static final String DOMAIN_NAME = "";

    public String getImage();
package org.dataparallel.example.jaxws;

import javax.jws.WebService;

public class ServiceEndpointImpl implements ServiceEndpoint {
    public String getImage() {
        return "aString";
package org.dataparallel.example.jaxws;

public class ServiceEndpointPublisher {
    public void publish() {
        publish(ServiceEndpoint.SERVICE_URI, new ServiceEndpointImpl());

    public void publish(String url, ServiceEndpoint service) {
        Endpoint.publish(url, service);

    public static void main(String[] args) {
        // single threaded testing service
        ServiceEndpointPublisher aPublisher = new ServiceEndpointPublisher();
package org.dataparallel.example.jaxws;

import javax.xml.namespace.QName;

public class ServiceClient {
    public void callService() {
        try {
            URL aWsdlURL = new URL(ServiceEndpoint.SERVICE_URI + "?wsdl");
            QName aQName = new QName(ServiceEndpoint.DOMAIN_NAME,ServiceEndpointImpl.class.getSimpleName() + "Service");
            Service aService = Service.create(aWsdlURL, aQName);
            ServiceEndpoint anEndpointInterface = aService.getPort(ServiceEndpoint.class);
            System.out.println("Service.getImage: " + anEndpointInterface.getImage());
        } catch (MalformedURLException mue) {

    public static void main(String[] args) {
        ServiceClient aClient = new ServiceClient();

Jax-WS has fallen out of favor for most new web services in favor of JAX-RS 1.1 or REST - it still has a place for specific SOAP clients or data types.

- in queue - see

Tuesday, January 31, 2012

iOS 7.0 Development

Updated: 20130303: Don't even think about using a MacBook Pro below the top 15 ich with SSD/16g-ram/2.6 Ghz.  Also there seems to be a defect in the original iPhone 5 where continous motion updates overloads the acceleration sensor resulting in a missing gravity vector - the iPhone 5s with the M7 coprocessor works fine for the 7 hours of 100ms granularity I have tested so far.

This is a really interesting time at Apple.
I just discovered that Objective C is partially based on Smalltalk.  Since I learned Smalltalk V as part of my first 2 years in university just before Java arrived - i needed to start developing in Objective C (for the iPhone, iPad and iPodt Touch) immediately.

1) Get a Mac - goto your nearest Apple store and ask for the Mac Mini to start.
2) Connect it to any keyboard, monitor, mouse
3) Set any http, https proxies
4) Get an ITunes account
5) Enter into an IOS developer agreement
6) Download Xcode
7) Get an iTunes Connect account and sign up for iOS Free and iOS Paid
     - fill out paid app legal forms
     - fill out banking info
8) Get some iOS and Objective C books
9) Start learning or transitioning to the unique syntax of Objective C
10) Leverage your knowledge of ORM and Rest
11) get apache Cordova / PhoneGap 2.2 and repackage your HTML 5 applications

You will need a Rest backend server - preferably on  Oracle Cloud, Google AppEngine or Amazon EC2.

101) A year later when your provisioning profile is near expiration - purchase a new one (I was not able to update it).
102) If you upgrade your device past what your iOS SDK is at (IE: iOS 6.1 on my iPhone 4s but Xcode 4.5 - not 4.6 for  6.1) you need to upgrade the XCode dev environment as well.
103) When you upgrade to using a proper Macbook Pro - install a new copy of XCode 4.6 and re-setup your provisioning and developer profiles.
104) fix for "valid signing identity not found" is to refresh in the provisioning profile tab of the organizer

105) renaming GUI elements - see p.54 of Beginning iOS 6 Development
ctrl drag ui element to controller.h file to name it and create a IBOutlet wrapper

My first hello world collate program in Objective C

int main(int argc, const char * argv[]) {

    @autoreleasepool {
        int64_t current = 0;
        int64_t maxValue = 0;
        int maxPath = 0;
        int path = 0;
        int64_t max; // 64 bit signed integer, like Java's long
        for (int64_t i=27; i<(1 << 30); i+=2) {
            current = i;
            path = 0;
            while (current > 1) {
                 if (current % 2 == 0) {
                    current = current >> 1;
                } else {
                    current = 1 + current + (current << 1);
                if(max < current) {
                    max = current;
            bool maxSet = false;
            if(maxValue < max) {
                maxValue = max;
                maxSet = true;
            if(maxPath < path) {
                maxPath = path;
                maxSet = true;
            if(maxSet) {
                NSLog(@"%lld: %lld: %i",i, max, path);
    return 0;

Total Pageviews