Tag Archives: OAuth

Better NodeJS OAuth example

Back when I was writing the Node-RED Alexa Home Skill node I used an example oAuth setup I found online here.

This worked for the initial setup but I couldn’t get Alexa to renew the oAuth tokens, as a temporary work around I set the token life to be something huge (300 years…). Again this worked, but it’s not the best and even thought I’m not likely to be around in 2318 to worry about it having crazy long token expiry times negates some of the benefits of the oAuth system.

I spent a couple of weeks bouncing email back and forth with the Alexa team at Amazon trying to work out what the problem was and I thought it would be useful to write up the findings to make life easier for anybody else wanting to use NodeJS/Passport to implement an Alexa Skill. I created a separate stripped down minimal skill to work through this without having to mess with the live skill and disrupting users and the whole thing is up on github here, but I’m going to walk through the changes here.

Let’s start by looking at what we had to start with, the following snippet of code is the part that returns a oAuth token to the remote service (in this case Amazon Alexa) when the user authorises it to use my service.

server.exchange(oauth2orize.exchange.code({
  userProperty: 'app'
}, function(application, code, redirectURI, done) {
  GrantCode.findOne({ code: code }, function(error, grant) {
    if (grant && grant.active && grant.application == application.id) {
      var token = new AccessToken({
        application: grant.application,
        user: grant.user,
        grant: grant,
        scope: grant.scope
      });
      token.save(function(error) {
        done(error, error ? null : token.token, null, error ? null : { token_type: 'standard' });
      });
    } else {
      done(error, false);
    }
  });
}));

This returns the absolute bare minimum (the 2 fields marked as required in the spec) for the oAuth spec.

{
  "access_token": "a1b2c3d4e5g6......",
  "type": "standard" 
}

The first fix is to add refresh token that can be used request a new token. To do this we need to add a new model to store the refresh token and it’s link to the user.

var RefreshTokenSchema = new Schema({
	token: { type: String, unique: true, default: function(){
		return uid(124);
	}},
	user: { type: Schema.Types.ObjectId, ref: 'Account' },
	application: { type: Schema.Types.ObjectId, ref: 'Application' }
});

And now to create a refresh token and add it to the token response from earlier.

server.exchange(oauth2orize.exchange.code({
  userProperty: 'appl'
}, function(application, code, redirectURI, done) {
  OAuth.GrantCode.findOne({ code: code }, function(error, grant) {
    if (grant && grant.active && grant.application == application.id) {
      var token = new OAuth.AccessToken({
        application: grant.application,
        user: grant.user,
        grant: grant,
        scope: grant.scope
      });
      token.save(function(error) {
        var refreshToken = new OAuth.RefreshToken({
          user: grant.user,
          application: grant.application
        });
        refreshToken.save(function(error){
          done(error, error ? null : token.token, refreshToken.token, error ? null : { token_type: 'standard' });
        });
      });
    } else {
      done(error, false);
    }
  });
}));

OK, so now we have a token response that looks like this.

{
  "access_token": "a1b2c3d4e5f6......",
  "type": "standard",
  "refresh_token":  "6f5e4d3c2b1a...."
}

This was basically what I was using when I launched the service, it has all the required fields and a refresh token. Amazon’s Alexa Smart Home API has an explicit error to return when a token has expired, so with that in mind I had assumed that when I return that error then the service would use the refresh token to get a new token. This assumption turned out to be wrong, even if you explicitly tell Amazon that the token has expired it won’t try to refresh it unless it is after the expires_in time in the token response… Now expires_in is listed as optional (but recommended in the spec) but it turns out that Amazon interprets a missing expires_in as tokens having an infinite life and as such will NEVER renew the token. To fix this we need to include an expires time. An expires time is already in the token model for the database (remember I’d already edited this to be 300 years) so we just need to get it included in the token response.

var expires = Math.round((token.expires - (new Date().getTime()))/1000);
refreshToken.save(function(error){
  done(error, error ? null : token.token, refreshToken.token, error ? null : { token_type: 'standard',  expires_in: expires});
});

Which finally gets a token response like

{
  "access_token": "a1b2c3d4e5f6......",
  "type": "standard",
  "refresh_token":  "6f5e4d3c2b1a....",
  "expires_in": 7776000000
}

This worked, the first time the token expired (in 90 days) but not the second time, because I’d not included a expires_in the token response when the token was refreshed.

As I said at the start all the code for a bare bones implementation is on github here, there are a couple of other changes, e.g.to make the system reuse existing tokens if they are still valid if the user removes and adds the skill to Alexa, to change the token type over to Bearer and including the scope information in the token response just to be complete. Should I ever get enough free time to work out how to get the model to work for a Google Assistant version of the Node-RED node this code will form the basis as that also needs an oAuth based service.

Moo Sticker engines fixed

A few years ago I create a web app to allow people to order their own sets of Moo stickers with the MQTT, Node-RED and Owntracks(at the time called MQTTitude) logos.

These worked well with people ordering about 1 pack a month until Moo changed the way they did authentication. I didn’t have time to fix it until today.

I’ve moved the app from being a J2EE app over to one written in NodeJS using express, this along with the change in Moo’s authentication method has made the code a lot shorter and easier to read. I’ve published the new code on github here.

To order a set of stickers click on the appropriate image:

MQTT Inside Sticker Engine

For a bit of fun round a project last year I created some Moo “MQTT Inside” stickers to put on the cases.

First sticker deployed

These became very popular around the office, with everybody wanting to label up their MQTT enabled projects and the first batch of 90 quickly disappeared.

Given demand was so high Andy Piper suggested it might be worth making it possible for others to order their own sets and pointed me at the Moo API.

The Moo API Doc is pretty good, but there aren’t any examples so it took a bit of trial and error. The API works by building up an object called a ‘pack‘. The Moo website uses the same API for the wizard you use to manually create any of their products and there is a very nice little Greasemonkey script called ‘Show API JSON’ which shows you the state of the pack object at each step. The API also lets you skip any steps in the process by just uploading a pack object already filled in. This meant I could reproduce the stickers by choosing to edit my original set and then using the script to save the pack.json file.

{
    "productVersion" : "1",
    "cards" : [
    ],
    "numCards" : 90,
    "extras" : [
        {
            "value" : "blue",
            "key" : "pack_colour"
        }
    ],
    "imageBasket" : {
        "name" : null,
        "immutable" : false,
        "items" : [
            {
                "cacheId" : "partner_interface_uploader:e34bb9c2-637b-558545db-4e61502d-ff34.png",
                "imageBox" : null,
                "imageItems" : [
                    {
                        "rotation" : 0,
                        "height" : 297,
                        "resourceUri" : "http://www.moo.com/is/o/e34bb9c2-637b-558545db-4e61502d-ff34.png",
                        "width" : 297,
                        "type" : "print"
                    },
                    {
                        "rotation" : 0,
                        "height" : 1024,
                        "resourceUri" : "http://www.moo.com/is/r/1024/e34bb9c2-637b-558545db-4e61502d-ff34.png",
                        "width" : 1024,
                        "type" : "preview"
                    },
                    {
                        "rotation" : 0,
                        "height" : 75,
                        "resourceUri" : "http://www.moo.com/is/t/75/e34bb9c2-637b-558545db-4e61502d-ff34.png",
                        "width" : 75,
                        "type" : "thumbnail"
                    }
                ],
                "copyrightOwner" : null,
                "resourceUri" : "filestore://image_original/e34bb9c2-637b-558545db-4e61502d-ff34.png",
                "croppable" : true,
                "shouldEnhance" : false,
                "type" : "front",
                "removable" : true
            }
        ],
        "type" : null
    },
    "productCode" : "sticker",
    "sides" : [
        {
            "type" : "image",
            "data" : [
                {
                    "imageStoreFileId" : "e34bb9c2-637b-558545db-4e61502d-ff34",
                    "resourceUri" : "filestore://image_original/e34bb9c2-637b-558545db-4e61502d-ff34.png",
                    "linkId" : "variable_image_front",
                    "enhance" : false,
                    "imageBox" : {
                        "center" : {
                            "y" : 12,
                            "x" : 12
                        },
                        "height" : 20.88,
                        "width" : 20.88,
                        "angle" : 0
                    },
                    "type" : "imageData"
                }
            ],
            "templateCode" : "sticker_image",
            "sideNum" : 1
        }
    ]
}

Resubmitting this with the ‘moo.pack.createPack‘ method and then using the ‘finish’ Drop in point to direct the user to Moo checkout page to pay for their new stickers.

I used a little library called scrib to do the oAuth and the whole thing can be done with just these few lines:

protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
	String authTokenString = request.getParameter("oauth_token");
	String authVerifyString = request.getParameter("oauth_verifier");
	
	Token authToken = MemoryStore.data.get(authTokenString);
	MemoryStore.data.remove(authTokenString);
	
	Verifier verifier = new Verifier(authVerifyString);
	Token accessToken = service.getAccessToken(authToken, verifier);
    
	String createPack = rootURL + "method=moo.pack.createPack&product=sticker&pack=" + pack;
	createPack = createPack + "&friendlyName=" + URLEncoder.encode("MQTT Inside", "UTF-8");

	OAuthRequest oAuthRequest = new OAuthRequest(Verb.POST, createPack);
    
	service.signRequest(accessToken, oAuthRequest);
	Response oAuthResponse = oAuthRequest.send();
	if (oAuthResponse.getCode() == 200) {
		System.err.println(oAuthResponse.getBody());
		JSONTokener tok = new JSONTokener(oAuthResponse.getBody());
		try {
			JSONObject json = new JSONObject(tok);
			JSONObject dropings = json.getJSONObject("dropIns");
			String finishURL = dropings.getString("finish");
			response.sendRedirect(finishURL);
		} catch (JSONException e) {
			e.printStackTrace();
		}
	} else {
		System.err.println(oAuthResponse.getBody());
	}
}

Andy is working to get a page set up on the mqtt.org site, but in the mean time you can click on the image bellow to order own set.

EDIT:

Andy has now added a Goodies page to the mqtt.org site that you can use to order stickers

Tracks2Miles Facebook logon

Ever since Tracks2Miles shipped users have been having problems authorising the app if they choose to log in with their Facebook credentials.

The problem is that once the user had logged into Facebook it would redirect to a blank white page.

Last night I had another look at what was causing it.

To start with I added a Log statement to the shouldOverrideUrlLoading() method to print out the URL that was being loaded. This implied that the last URL loaded was from Facebook:

http://www.facebook.com/connect/uiserver.php?method=permissions.request&app_id=xxxxxxxxxx&display=touch&redirect_uri=https%3A%2F%2Fapi.dailymile.com%2Flogin%2Ffacebook%2Fcallback&state=next%3D%252Foauth%252Fauthorize%253Fresponse_type%253Dtoken%2526client_id%253Dxxxxxxxxxxxxxxxxxx%2526redirect_uri%253Ddm%253A%252F%252Flocalhost%252Ffoo&type=web_server&fbconnect=1&perms=email%2Cpublish_stream%2Coffline_access&from_login=1&refsrc=http%3A%2F%2Fm.facebook.com%2Fsetup_machine_mobile.php&refid=0&m_sess=MUm3C-eKFY7K8h&_rdr

I poked around at this for a while, but didn’t get very far, until Chris Banes pointed me at the source for the facebook-android-sdk, specifically the FbDialog.

When I added some trace to the onPageStarted() method I made some progress, the output from this implied that Facebook page had actually loaded successfully and redirected back to dailymile.

https://www.dailymile.com/login/facebook/callback?code=HphJjEM9nKsSez0ZW28W-FcvN80xR049KtQtk7CrykM.eyJpdiI6ImxGNVEyQ0FuSlF1X0xKdWJoU1MtZncifQ.yaVPEGNBOBPMQtZjrE431mw6a4-6yXUrvz0XrCd2ub44KKArQXpfiydYvhocYZuV0FW9mIO_3KLjBEO1XejL-EoplURcmbpoA3ueXjQJmvsu9Gf0Vaj7CLOdLOOpFw-d&state=next%3D%252Foauth%252Fauthorize%253Fresponse_type%253Dtoken%2526client_id%253Dxxxxxxxxxxxxxxxxxxxxxxxx%2526redirect_uri%253Ddm%253A%252F%252Flocalhost%252Ffoo

I had tried loading this new URL with Firefox on the desktop and immediately got a certificate mismatch error. The URL was hosted on www.dailymile.com but the cert was for api.dailymile.com

I was a little surprised that default WebView just swallowed this with no error messages, but a bit of googling and found the following blog post about how to work round the certificate miss match and even how to make it work with Android builds before 2.2

I also pointed the problem out to Ben from dailymile and he has fixed the cert mismatch with in a couple of hours.

There will be an updated version of Tracks2Miles in the next few days with these changes (just to be safe) and a couple of other updates around distance units.

My Tracks to dailymile GPX route exporter – part 2

In my last post I talked about a project I’m working on to export GPS tracks from My Tracks directly to dailymile.

I was having problems getting the mobile client flow for oAuth 2.0 to work with dailymile. As a work around I decided to see if I could get the User-Agent flow to work for this application.

At the end of the previous post I suggested using an Intent filter to catch a redirect to a custom URI schema, while I got this to work it was messy as it left the browser app on the activity stack during the authentication. While looking for a way round this I found the documentation for the WebView widget and came up a much better solution. The WebView widget allows you to embed HTML content in your application so it should be possible to embed the web based authorisation set into a Activity.


<pre>
getWindow().requestFeature(Window.FEATURE_PROGRESS);
WebView webView = new WebView(this);
setContentView(webView);
final Activity activity = this;
webView.loadUrl(Constants.AUTH_URL);
</pre>

In the onCreate method of the Activity I have done away with a normal layout and just used a full screen WebView and asked it to load the authentication URL from the User-Agent Flow

https://api.dailymile.com/oauth/authorize?response_type=token&client_id=<CLIENT_ID>&redirect_uri=dm://locahost

Login Authorise

The callback redirect URI (dm://localhost) still has a custom schema similar to the one suggested in the previous post. Assuming the user allows access to their dailymile profile we can intercept the redirect by overriding the WebViewClient on the Webview which will look a bit like this:

dm://localhost#access_token=[ACCESS_TOKEN]

webView.setWebViewClient(new WebViewClient() {
  @Override
  public boolean shouldOverrideUrlLoading(WebView view, String url) {
    if (url.startsWith("dm://")) {
      String summary = "<html><body>Grabbing Account info</body></html>";
      view.loadData(summary, "text/html", "utf-8");
      String token = url.substring(url.lastIndexOf("=")+1);
      User user = Utilities.getAccountDetails(token, activity, handler);
      if (user != null) {
        user.setToken(token);
        finishLogin(user);
      }
      return false;
    }
    view.loadUrl(url);
    return true;
}});

Now I’ve got the AuthToken I can tuck it away in the Android AccountManager so I can grab it later when I want to upload a training session.

Twitter2MQTT bridge

The recent switching off of Basic Auth by Twitter meant I had to rework some of the applications I have running at home. One of these application bridges Twitter to my local MQTT broker so that any DMs and mentions get published to two topics and also a topic that updates my status to what ever is posted.

The original version of the application just made basic http requests and parsed the XML responses, rather than just try and bolt on OAuth support I thought I would be better to start again and use one of the existing libraries that drive the Twitter APIs. There is a list of libraries for different languages http://dev.twitter.com/pages/libraries, I had a bit of a look at a couple of them and settled on Twitter4J.

In order to use OAuth you need to register your application with Twitter, you can do that here http://twitter.com/apps/new. Once registered you will get Consumer Key and a Consumer Secret. Because Twitter are using these keys to help to cut off spammers, keys need to be kept secret in order to prevent spammers knocking legitimate applications off-line, If you want to build the code attached here you will need to apply for your own key.

Before you can use a Twitter application you need to authorise it to act on your behalf, this is a 3 stage process.

  1. The application creates a URL based on it’s Consumer Key and Consumer Secret.
  2. The user follows the URL and signs into Twitter and is then asked if they want to allow the application to access on their behalf. If they allow the application then Twitter supplies a PIN.
  3. The user passes the PIN to the application which uses this to retrieve a Token, This Token is used to authenticate the application when ever it needs to act on the users behalf.
Twitter Application Authentication
Twitter Application Authentication

To do this with Twitter4J you need to do something like this:

    Twitter twitter = new TwitterFactory().getInstance();
    twitter.setOAuthConsumer(consumerKey, consumerSecret);
    try {
        RequestToken requestToken = twitter.getOAuthRequestToken();
        AccessToken accessToken = null;
        BufferedReader reader = new BufferedReader(
                                       new InputStreamReader(System.in));
        while (accessToken == null) {
            System.out.println(requestToken.getAuthorizationURL());
            System.out.println("Please follow the URL above," + 
                      " enter the PIN provided and then press Enter");
            String pin = "";
            try {
                pin = reader.readLine();
            } catch (IOException e) {
                e.printStackTrace();
            }
            accessToken = twitter.getOAuthAccessToken(requestToken, pin);
        }
        String token = accesToken.getToken();
        String secret = accesToken.getTokenSecret();
    } catch (TwitterException e) {
        e.printStackTrace();
    }

The token and the secret should be stored so the user doesn’t need to authorise the application each time it starts. Now the application is authorised you can post status updates and access the DM and Mention streams like this:

    List<DirectMessage> dms = twitter.getDirectMessages();
    List<Status> mentionsList = twitter.getMentions();
    twitter.updateStatus(newStatus);

Now that I can access the updates I need to publish them to topics and listen for when a new status is published. There is a Java library for accessing an MQTT broker provided by IBM known as the IA92 package. This package provides both J2SE and J2ME versions of the libraries. To create a connection to the broker

    IMqttClient client = null;
    try {
        client = MqttClient.createMqttClient("tcp://brocker.loc:1883"
                                                            , null);
        client.connect(clientID, true, (short) 100);
        client.subscribe({"Twitter/send"}, {1});
        client.registerSimpleHandler(new MqttSimpleCallback() {
            public void connectionLost() throws Exception {
            }

            public void publishArrived(String topic, byte[] payload, int qos,
                            boolean retained) throws Exception {
                twitter.updateStatus(new String(payload));
            }
        });

    } catch (MqttException e) {
        e.printStackTrace();
    }

Each client needs a unique id, or the older client will be kicked off when the new one collects, if you want to run multiple versions of the bridge then it may be a good idea to append the OAuth access token or the twtter screen name to the clientID.

Taking all that and knocking some of the rough edges off I have an app that will publish DMs on Twitter/<screenName>/dm & mentions on Twitter/<screenName>/mention and listens on Twitter/<screenName>/send for status updates to publish. There is a second app which subscribes to the mention and DM topics and forwards these messages on to the topic that sends SMS messages to my phone.

Next on the list of additions is a filter that will forward photos attached to tweets as MMS messages and probably some support for bridging search terms as well.

You can download my code from here, remember you will need your own Consumer Key and Consumer Secret to build it yourself, but there is a pre-built version with a valid key included.

Twitter2MQTT.tar.gz

Resources