August 30, 2012

We’ve finally made it – from bucket to bokashi to 3 compost bins

This week we reached a major milestone in our business – something we’re all very excited about. Finally, after years of hard work and dedication, and several wrong turns along the way, we’ve got there.

We’ve just bought our second compost bin, to sit alongside our existing bin and worm farm.

To put this in perspective, some history. Like most good things, it starts with coffee.

Back in the day, when we were working in a ridiculously large but trendy warehouse space on Randle street, we had a bucket. We needed the bucket once we bought our first coffee machine, and then realised we needed somewhere to put the grinds. However, our trendy space was so cool and hip that it lacked a sink- or a kitchen. So we settled on a bucket.

The bucket’s demise began when people started emptying out the water from the coffee machine, and the bucket began to get more liquid and contain all manner of scraps. This went on for a while, and the bucket became more and more disgusting.

Things came to a head when I started finding teaspoons in our toilet. I was sure someone must be shooting up in there – I wasn’t really sure how to deal with that from an HR perspective (did we say anywhere in our contracts that it wasn’t cool to shoot up during lunch?). No one seemed to be nodding off at their computers or acting odd.

It turned out that our cleaner was tipping the entire bucket down the toilet and then flushing it. And occasionally there was a spoon in the bucket. Everything else was flushed away, but the spoon remained. Those guys don’t do our cleaning any more.

Our next office had a kitchen, but we needed somewhere for food scraps and coffee grounds. So on the advice of a friend, we got a Bokashi. For those of you unfamiliar with the Bokashi concept, the idea is that you have this completely sealed small tub that you put food waste into, and then sprinkle some special magic Bokashi powder over, and the scraps get dissolved by all the special bacteria in the Bokashi powder. Or something like that. The supposed advantage being none of the smell of compost and good for small spaces. Sure.

Things didn’t quite work out like that – maybe our powder was out of date or something. We filled the tub in a week or so, and pretty soon the scraps turned into this dark grey gloop with the most incredible stench. Opening the lid to put more food scraps in would release a warm gush of fetid stink, so pretty soon everyone learnt not to open it under any circumstances. I tried tipping a tiny bit out, diluting it with water and putting it on some of our office plants. The next morning, all of our plants were dead.

So the Bokashi sat there for about a year, no one quite sure what to do with it. We were worried that moving it might cause an explosion or toxic leak. But the longer we left it, the worse it got.

Finally, our problem was solved when we got our office repainted. After the job, the painter left some of his cans of paint in our kitchen, and then turned up with a mate a few days later to collect them. It seemed they had decided to have a smoke of the peace pipe, and they turned up extremely stoned. They carried out all of their paints, and loaded them into their van and drove off very very slowly. With our Bokashi.

So then we decided to get serious. No more messing around with buckets or crazy systems – worms were the answer. We got a worm farm. The worms are little champions, but with limited capacity. With 28 people we make about 8 small buckets of scraps a week. And when that filled up we got a big compost bin to deal with extras. Now that has finally filled up, so we’re onto our second compost bin.

But somewhere out there, sitting in the back of a paint splattered van amongst some cans of paint, sits the ticking time bomb of our Bokashi.

August 3, 2012

Which Shopify account is the best value for money? It depends of course…

As part of a recent assessment for a client e-commerce solution we looked into Shopify. Part of that meant checking out their prices and finding the account level which would hit the sweet spot for the clients needs. In the end we didn’t go with Shopify, but this price comparison of their accounts is pretty handy to have on hand.

The graph above charts monthly sales to monthly costs for all four account levels at Shopify. What it gives us is a pretty straight-forward answer to which account is best for anybody who is planning on using the service.

  • If your sales are below $3,000 per month, stick with the “Basic” plan.
  • If your sales are between $3,000 and $24,000 per month, then the “Professional” plan is for you.
  • If your sales are above $24,000 per month then the “Unlimited” plan is the best value.

There are other reasons to switch to higher level accounts, like to get more storage and SKUs, but if price is the driving factor then this simple analysis covers everything.

Why were we looking at Shopify, and why didn’t we use it?

We often have a requirement to build some kind of shop or ecommerce integration in to a site. Clearly different projects handle this differently- for a site like 12WBT, the ecomm bit isn’t really a separate shop area – it needs to be very tightly integrated into the membership process. So we built that. Other projects like beautyheaven have a separate “shop” area, which means building something isn’t necessarily the best approach.

A quick summary of the options:

  • Build something that suits requirements. Advantage is that it can fit exactly. Disadvantage is that you might have just reinvented the wheel – ecomm has been around for a while now…
  • Build upon an existing open source shop- taking a project like Spree and integrating this. Advantage is that you get lots of functionality out of the box, and an ability to tweak as required. No one is clipping the ticket. Disadvantage is that the more you tweak, the longer it takes in dev time (apologies for stating the bleeding obvious).
  • SAAS – skin or theme a SAAS system like Shopfiy- advantages being the shop can be up in minutes, and no hosting to worry about. The API allows for some integration into your app. Disadvantage is that any limitations to the API limit what you can do, and you’re paying per transaction- which might not suit some projects.
  • Install something commercial – Advantages are that you might find a tool with the exact shop functionality you want- in which case happy days. Disadvantages are that it might not fit your actual requirements

Two examples of where a SAAS approach like Shopify didn’t work for us

  • Coupons- we needed to dynamically generate a coupon code in the main app, and then send that to someone so they could use that in store
  • Points/Reward System- we needed to award points to members based on activity in the app. They could then redeem these points by “buying” stuff in the store.

That said- if what you’re after is a site with the main function of selling stuff, you’d be crazy not to consider a SAAS approach. And among the SAAS options, Shopify is certainly a very good option – well built, reliable and a well thought out & developer friendly theming system.

June 17, 2012

Going mobile: some options to address increasing mobile traffic to your site

traffic

Each month, each of our sites seem to be getting more and more traffic from mobile devices. Not so long ago, mobile was an interesting but small segment (“who on earth would use their phone to browse the site? They must be stuck on a bus bored somewhere”).

What started at 5% grew to 10% the next year, then doubled again, and now some of our sites are over 40%. Email is even more interesting – checking an email from your phone is now pretty easy, which drives the mobile vs PC rate right up. Campaign Monitor is predicting this month over half of their emails will get opened on a mobile device.

So mobile traffic is getting bigger and bigger, and everyone is feeling the need to do something. But it can be a bit confusing as to what actually to do – a common reaction seems to be to build a mobile application, but this isn’t necessarily the most appropriate solution.

Let’s start with a basic premise: providing a better experience for users on mobile devices is good. It’s something you want.

There are four approaches for improving the mobile experience. These aren’t exclusive – as in you don’t need to pick only one. In fact, our recommendation is to adopt all four if you can.

1. Check your existing site

The current generation of smart phones have pretty good browsers, so your site should work OK on them (albeit small). If it doesn’t, then it is probably an indication that some updates are required anyway. I suppose what I am saying is that if your site doesn’t work on a mobile properly now (excluding things like Flash and file uploads) then address that first, as your site probably breaks on other non-mobile browsers now.

2. Responsive site layout

A responsive layout reorganises itself depending on screen size. So the actual page that each device (PC or mobile) gets is the same, but the layout changes depending on size available. On a wide screen the navigation bar might be at the top, with an image below it. The same page when seen on a smaller screen might change to be text only with no image under it. There might be a few stages in between “big” and “tiny” to allow for mobiles or tablets with slightly bigger screens – eg: same navigation bar but no image.

The advantage of responsive is that it “just works”- one layout can intelligently resize itself for different sizes. As bigger mobile screens come out, your responsive layout will adapt.
The disadvantage is that mobile devices are getting the same page – even though they can’t see some of it. This means the page can be bigger in file size than it needs to be, which slows things down.

3. Separate mobile pages

Separate mobile pages means that there is a different URL for some or all of the pages. When a user arrives on the site, mobile users are redirected to these separate mobile pages.

The advantage of this is that your page can be crafted just for mobile users- there might be some features that just don’t make sense on a mobile, so you can drop those. There might be other new, mobile specific functionality that you can include just for mobiles. Having separate pages means page size can be optimised- particularly important for mobile as they tend to process and render pages more slowly. So a big plus for the separate page approach is speed.
A disadvantage of separate pages is that you’ll now need to maintain a separate version. Whilst this is the case, our experience has been the effort required to maintain separate mobile pages (using say JQuery mobile) and a responsive layout is very similar.

4. Mobile app

Having a mobile app involves building a separate application, and then getting this published in either the android or apple itunes store. The advantage here is that rather than just being another URL, your site is now something that lives on the user’s desktop. They can run your app directly from their mobile rather than going to the web first.

The downside is the effort and cost of development and publishing. Whilst I realise there are lots of sites claiming this is easy and simple, the unfortunate truth is that it takes a while to do properly. An app needs to be written in a different programming language to your site, and the publishing process is more involved. Once published, your app will live or die based on downloads and reviews. Bomb on these and popularity will plummet pretty quickly, so it is worth doing properly the first time. Do it on the cheap and all you’re doing is blowing that money and setting yourself back another few months.

March 30, 2012

Bypassing the sign up hurdle with super-soft joins

Red Ant client NCPIC has been pushing the boundaries of web-based psychological interventions for the past four to five years. Their latest intervention program Clear Your Vision had a few hurdles it had to overcome when it came to translating it to the web.

Clear Your Vision homepage screenshot

The intervention is aimed at young people, generally of high school age, and the topic we are wanting them to engage in is their own drug use, specifically their cannabis use. Obviously a web site belonging to a government funded agency that is offering a place to chronicle a person’s drug use activity is not a place where we are going to easily get people to provide their email address.

The answer for us was to bypass sign ups and allow anybody to start the intervention process. During the course of the program we consistently offer a “save” functionality.

When selected, this gives a visitor the option of either providing their email address or downloading a personalised PDF without providing an email address.

Either way, they obtain a unique keyed link for their session. Clicking on the link brings them back to where they left off with the key in the link forming the only authentication. The email path is effectively a “soft join”, but the PDF path is a “super-soft join” as it requires zero credentials.

With the PDF option, we have no way to directly identify the person, and so we can allow people who do not want to provide their email address to still utilise the resource. Those that provide their email get the benefit of being automatically reminded via email when they should return to complete the second part of the program.

The authentication is “one factor”, but where the usual factor would be “something the user knows” – like a password, our factor is “something the user has” – their keyed unique link. By nature this is less secure certainly – as we rely on the user to keep it secret – but this level of authentication is OK for this application because in no way do we reveal any personally identifying data or offer the ability to reset the login credentials via the user interface. If you are offering the ability to change the account settings or are displaying personally identifying data, then this is definitely not a good approach for your application and you should probably use a normal password bound authentication method.

March 28, 2012

Manage SSL redirection in Nginx using maps, and save the universe

We use the Nginx web server for the majority of our websites at Red Ant. It’s incredibly flexible, and built for speed (freakishly fast in fact). One drawback though is that finding solutions for some of the more curly configuration problems can be difficult. Nginx is relatively new and has had a fairly rapid release schedule up to version 1.0, which has led to a moving target for documentation. It was also a while before all the original basic documentation was translated from Russian into English, which doesn’t help mono-lingual people like me. With this knowledge, I was not surprised when I couldn’t find any solutions out there for my specific problem:

The problem

We need to be able to enforce SSL connections (HTTPS) on some URLs. We also need to be able to enforce PLAIN connections (HTTP) on others. Further, there are some URLs which we want to just stay with whatever protocol they were requested on so they won’t redirect at all.

One approach is to use a great big series of rewrite rules, but that would be pretty unwieldy, would involve a bunch of regular expression variable capturing, and likely also mean using more than a couple of “if” blocks (which I have been told are evil and slow, even if few people have the sort of traffic where that really matters). We also may have to maintain them across two virtual hosts, making it hard to check for conflicts between the two lists. In cases where the virtual host pulls double duty serving HTTP on port 80 and HTTPS on port 443 simultaneously, we would have to check the protocol first as well, further confusing the config. There has to be a better way.

We already use Nginx maps to create standard redirect lists on some sites, but mostly these are static lists which are used to redirect old URLs to new ones. We do this so that when we rebuild sites using Ruby on Rails they don’t loose the Google juice that is already attached to all those horrible, horrible default.aspx?foo=bar&578432754230 URLs…

Yes, I’m looking at you .NET developers.

The wiki page on Nginx maps gives a basic example of using a map to redirect request URLs to new locations. But that is no good for us, because we explicitly want to redirect only if the protocol is not our preferred one. We also only want to redirect to the same URL, albeit over a different protocol. A basic map like that will send our client into a redirect loop. We would need two separate maps and that kind of takes us back to square one. There is also potentially a problem with making matches too broad in separate maps and causing redirect loops again, whereas a single map will match only one value. Using one map we can also take advantage of the map “default”. It’s complicated, so just trust me — separate maps = do not want!

The solution

Rather than build a map of request URIs to redirect locations, we are instead going to build a map of request URIs and their preferred protocols — “http”, “https”, or “none”.

Here’s what a really simple example of that might look like:

map $uri $example_org_preferred_proto {
	default "http";
	~^/(images|css|javascript)/ "none";
	~^/sign-up/ "https";
	~^/account/ "https";
}

The first line defines the map block. The first variable $uri is what we are checking against — in our case the incoming request URL without the protocol, host name or query string. The second variable $example_org_preferred_proto is what we will map the result of the match to.

The second line is a special “default” case inside the map. Most of our URLs should always be HTTP, so if none of the patterns in the block match the requested URL, then the result will default to this value of “http”. You may want to default to one of the other values depending on which will save you more rules and give you the shortest list.

The third line is matching our static assets directories which we need to be available over both HTTP and HTTPS. So we assign them a value of “none”. Since Nginx 0.9.6 you can use regular expressions inside maps. The tilde character “~” at the start of this line indicates that this is a regular expression. These work like other regular expressions in Nginx… which I won’t explain here.

The fourth and fifth lines make sure that our sign ups and account management pages are always HTTPS. This could have been done in one line like the previous one, but part of the point of doing this is to create logical and readable groupings of URLs to ease ongoing maintenance. In a much longer list, this will matter more.

Inside our virtual host configuration we can apply the map like this:

map $uri $example_org_preferred_proto {
	default "http";
	~^/(images|css|javascript)/ "none";
	~^/sign-up/ "https";
	~^/account/ "https";
}
server {
	listen 80;
	server_name example.org;
	root /var/www/example.org/htdocs;
	if ($example_org_preferred_proto = "https") {
		return 301 https://example.org$request_uri;
	}
}
server {
	listen 443 ssl;
	server_name example.org;
	ssl_certificate /var/ssl/example.org.crt;
	ssl_certificate_key /var/ssl/example.org.key;
	root /var/www/example.org/htdocs;
	if ($example_org_preferred_proto = "http") {
		return 301 http://example.org$request_uri;
	}
}

Note that the map is defined outside of the server blocks. That’s one of the advantages of this method over a series of rewrites — the map is setup when Nginx starts and is available globally throughout the configuration. This is supposed to improve performance but I don’t know why exactly as the mapping itself occurs on demand. I’m happy to remain ignorant of the specifics here. It also is a bit easier on the readability of your logs if you are debugging your Nginx configuration as it outputs less debug lines.

The real payoff is in those if blocks, we simply query the preferred scheme for this request and if it doesn’t match, then we redirect to it. Here I’ve used permanent redirects by indicating “301″, but during testing it is wise to use temporary redirects by indicating “302″ instead.

But I’m behind a load balancer which handles the SSL, so this won’t work for me!

True, very true. Some people are behind load balancers which do SSL termination for the incoming requests. This means that they are usually utilising a single server block for their virtual host configuration and that virtual host is set to only listen on port 80, because everything being passed to it is in PLAIN/HTTP.

For these cases we need to make a couple of adjustments to use the X-Forwarded-Proto header that is usually sent by the load balancer. By doing so we can detect the incoming request’s original protocol, a task which was done automatically in the previous example due to the separate server blocks listening on different ports.

map $uri $example_org_preferred_proto {
	default "http";
	~^/(images|css|javascript)/ "none";
	~^/sign-up/ "https";
	~^/account/ "https";
}
server {
	listen 80;
	server_name example.org;
	root /var/www/example.org/htdocs;
	# If there is no preferred protocol, then prefer the current protocol
	if ($example_org_preferred_proto = "none") {
		set $example_org_preferred_proto $http_x_forwarded_proto;
	}
	# Redirect if the forwarded protocol doesn't match the preferred scheme
	if ($example_org_preferred_proto != $http_x_forwarded_proto) {
		return 301 $example_org_preferred_proto://example.org$request_uri;
	}
}

We have to add an additional if block here so that our URLs that prefer “none” don’t go into redirect loops. It simply re-maps the URLs preferred protocol to whatever the forwarded protocol was. For those URLs, the next if block will never be true, so no redirect occurs for “none”. Great!

It might be possible to map the “none” URLs directly to $http_x_forwarded_proto in the map itself, but my guess is that doing so would be slower for large lists due to the variable requiring expansion several times.

In the second if block we simply redirect to the preferred protocol if it doesn’t match the forwarded protocol from the load balancer. That covers both the remaining “http” and “https” cases.

If you aren’t behind a load balancer, but use a single server block listening on port 80 and 443 for HTTP and HTTPS traffic, you can probably use a similar method, substituting $http_x_forwarded_proto with $scheme.

If you have a big map, then you may hit Nginx’s memory limit for map hashes. This can be increased using the directives map_hash_max_size and map_hash_bucket_size. In any case, Nginx will tell you what’s up when you try to restart it.

If you have any questions specific to your setup, or any suggestions for improving these methods then drop a comment here.

Newer Posts
Older Posts