Race conditions

I’m doing a lot of phone development recently. The biggest change that this has made to my style of coding, is that it means I’ve had to learn asynchronous coding methods, which I didn’t have to do when my code was mostly server-based.

For instance, on a server in PHP, when you retrieve a database item, the retrieval is “synchronous”. This means that when you run something like the following pseudocode, line 2 will definitely happen after line 1 has finished:

1. get data from database
2. print data to screen

However, the local databases in HTML5 (indexeddb) is asynchronous. This means that you make a request, and it goes off on its merry way and might not come back for a while, and meanwhile, your program continues immediately to the next line.

In asynchronous coding, we have a problem called “race conditions”. The problem is that we cannot be sure when the asynchronous code will be finished its job, so need to think outside the box in order to make our job bullet-proof.

Using the pseudocode above, line 2 will be reached alright in an asynchronous system, but it will very probably fail, as line 1 will probably not yet be finished.

The solution in a simple case (retrieve one item, then do something), is to use a “callback” function. This is a function that the code is to run after the retrieval has finished.

We can rewrite the first example like this then:

1. get data from database, with the following callback:
    1. print the data to the screen

That’s a simple case, and easy to understand.

File access in HTML5 is also asynchronous, and throws up another problem.

With a database, you can write a query such that multiple rows of data are returned at the same time. But with file access, if you want to get the contents of multiple files, you need to access each file independently. The problem is that each of these is a new asynchronous process.

The solution is to have a count-down, which you set to the number of items to be retrieved, and as each item is retrieved, decrement the count-down. If it hits 0, then you have all the information you wanted and can call the callback.

Here’s how it would look in synchronous code (on a server, for example)

1. foreach file, retrieve the file
2. do whatever is next

and here is the asynchronous version:

1. set $leftToGet to the number of files to retrieve
2. foreach file:
    1. retrieve file, and when it's retrieved:
        1. reduce $leftToGet by 1
        2. if $leftToGet is 0, then do whatever is next

I know it looks really simple when I write it like that, but when you’re used to synchronous code, its not obvious at all!

newsletter, november 2011

A copy of the newsletter just sent out, detail improvements to the KV Sites CMS over the last two months.

Read more »

All website payments are now credits-based

I hate it when I receive large bills, as usually happens when there is a period of months between each bill.

When a large bill comes in, the following things happen to deal with it:

  1. panic.
  2. promise myself I’ll deal with it when I have the money to spare.
  3. forget.

And then the demand letters start.

At KV Sites, we thought the way to solve this problem was to instead ask for people to pay monthly. The smaller fee would be easier to pay, and it should be no big deal to just pay.

Unfortunately, that has a different effect:

  1. relax – it’s only €10.
  2. promise to pay it tomorrow when you can be bothered.
  3. forget.

We still believe that the smaller, more frequent, fees model is the right answer, but we recognise that some people see the fee as so inconsequential that they simply stop paying.

So, we’ve automated our process completely.

Now, all our customers are on a Credits system, where recurring fees occur each month, and if the account doesn’t have enough credits, then the website is simply turned off. Just like that.

And because this is a post already full of lists, let’s list the benefits to the client:

  1. you get a week in which to make sure the credit is available (only €10!).
  2. you can pay without leaving the comfort of your chair, using the PayPal button in the Credits section of your administration area.
  3. if you prefer to pay yearly, then simply purchase a year’s worth of credits.

And the benefits for KV Sites:

  1. less headaches calling and asking for small amounts of money to please (please!) be paid.
  2. constant automated income that we can spread more evenly through the month.
  3. we now have more time available to work on the engines themselves, to improve them for you!

If you’re not already a customer of KV Sites, consider it! We offer very affordable websites, either built by us from your content (€300 setup, €10 monthly hosting), or built by yourself (€20 setup, €10 monthly hosting, first three months hosting free).

20eurowebsites.com

As I said in a recent post, one of the things we love to do at KV Sites is automation.

To that end, we’ve come up with a way of automating our website creation to the point that we are not involved in it at all:

20eurowebsites.com

In that site, you fill in a form, pay €20, and a minute later, you own a content-managed website, including its domain name.

Because of the total lack of our involvement, we were able to waive our usual setup fee (€400) and charge pretty much the price of the domain name.

Of course, you still need to pay hosting. Giving that away for nothing would be like giving someone a free car and them expecting you to also pay for the petrol they use. We charge €10 per month for hosting, which is charged automatically every month.

We are still interested in your business, of course – the 20eurowebsites.com is for people that are comfortable setting things up themselves, and for everyone else, we are still happy to help out at our normal low prices.

new team member

KV Sites welcomes Conor Mac Aoidh to the programming team.

Conor will be working with us for four months, purely on our CMS. He’ll be reporting on updates at our forum.

We’ve known Conor for years, since he briefly interned in Webworks about 3 years ago. He showed that he was a very quick learner, and not afraid to take the initiative and do cool and fantastic stuff at the drop of a hat.

Conor is experienced in PHP and jQuery (along with all those other little HTML/CSS/MySQL bits that go with the territory), and will be a valuable member for the next few months.

Afterwards, he’s back to UCD to do something or other.

We’re looking forward to his first project – Conor will be creating a themes server, so we can keep all the WebME themes in one central location.

Conor’s blog is at http://blog.conormacaoidh.com/.

Living in the cloud

In this article, I’ll discuss a data-loss problem I had over the weekend, why it doesn’t matter in the end, and how to avoid it in the future.

Over the weekend, I managed to break my laptop. An operating system upgrade went wrong, and when I went to re-install, the operating system totally cleared the contents of my drive, despite me carefully choosing options that should have done the reinstall and left everything else alone. (thank you Fedora)

I’m not really all that angry though. I was worried for a few minutes that I had lost a load of important stuff, but all I can think of that I’ve lost is a budget forecast, and some work I was doing offline before placing it online. In all, about two days of work is gone. That’s not bad – it could have been much worse!

Taking stock, here’s how I use my computer:

Email

Every email account that I have is loaded through IMAP. This means that the email is kept on the server, so that if I ever need to read the email from another computer (or have a hard-drive crash!) then I can do that.

The alternative is POP3. This was the default for most people for years. In this case, email is downloaded to the computer and removed from the server. The reason this was done was that in the old days, bandwidth was slow, so it was better to download your email so it was quicker to load. And, server space was expensive, so if you kept your emails online, you’d have to pay a bill to the ISP.

The problem with POP3 is that if you lose your hard-drive’s contents, then you also lose all your emails.

Source Code

I’m a programmer. All my work involves either writing code, or doing something manually and then figuring out how to automate it later.

Luckily, my work is based online anyway. As a PHP programmer, everything I do lives on a server online, except when I’m writing it and testing it offline.

There are three ways that my code is safe:

  1. It’s kept on a server that’s not near me – that way when I destroy something locally (oops, coffee spill!), it’s not affected.

  2. All my code is open source, and the source code is available in Google’s repositories (KV WebME, KFM, SaorFM).
  3. My online server is backed up every single night to a hard-drive in my house.

Task lists

All companies have some sort of task list that they keep current – whether it’s in a notebook, or a word-processor document, etc., there’s always a list somewhere.

Personally, I have two kinds of task – day to day tasks, and “bugs”.

The day-to-day tasks are tasks such as “check the DNS for this site”, or “move email to that server”. One-line descriptions of items where there is little-to-no discussion needed for it, and usually it’s just an internal “want” list. I keep these in an account at RememberTheMilk.com. As an aside, I love the interface on that thing!

For larger tasks, which require feedback from either other company members (none as yet ;-) ) or from the public or resellers, I keep a copy of Mantis Bug Tracker on one of my servers.

Accounts

KV Sites is only a small company so far, so I’ve no need for anything too fancy. My method of accounting is actually to gather up all my receipts and invoices in a pile, and hand them to my accountants. They’ve never complained about this!

When I need to create an invoice, I again use an online service. I have a copy of Simple Invoices which I have on a server.

Heh – and as of yesterday, I use Google Docs for my budget forecasts.

Backups

At home, I have a media centre running XBMC, to which I’ve attached a 1TB hard-drive specifically for backups.

Every night, the entire hard-drive of my online server is backed up to that external drive. I keep the last seven days of backups, and one backup from evey week before it.

Conclusion

Because I have all my business processes online, I haven’t really lost all that much.

In a way, this was a “wake-up” call. It was yet another reminder that if you keep all your eggs in one basket, some day that basket will drop.

Solution

To keep me on my toes, I’ve decided that every single month, on the 1st, I’m going to completely wipe my laptop and reinstall everything.

This serves two purposes:

  1. Installing from scratch means I always have a nice fast machine. Upgrading from older versions of operating systems generally makes the machine slow down eventually.
  2. Wiping the hard-drive forces me to make sure the data on the drive when I do that is either non-essential, or already backed up somewhere else. The ideal is that there is never any essential data on the machine in the first place.

websites vs web apps

At KV Sites, we currently focus on providing websites to our customers.

However, this is not our main goal. Creating websites is a means to an end. We want to eventually enable all of our customers to create their own websites, and to eventually become creators of websites for their own clients.

How we do this, is to concentrate on automation.

Whenever we are asked to do anything, we do it, but we try to do it in such a way that the next time we are asked to do it, we can simply press a button.

The obvious example is the creation of a website. To build up a CMS-based website manually involves a number of technical points – create and install the database, create the virtual host, install the CMS, install the template, fill in the content. Purchase the domain name, configure the DNS

Every point above can be automated except the “fill in the content” part, and so we have automated it.

For our company, creating a website involves us simply clicking a “Create a new site” button:

This installs a CMS, including the database, a template, and even some sample content so it doesn’t start out blank.

All that leaves is for the client’s company-specific content to be entered, and for them to choose a domain name, and a design template change if they want it.

This is the reason why we are able to provide our websites much cheaper than most traditional web design companies – we try to cut out as much of the “behind the scenes” work as we can, so we don’t need to charge our clients for that work.

Now, back to the topic…

As said at the beginning of this post, our main goal is actually not to be a web design company.

The goal of the company KV Sites is to become a web applications provider.

While we still love our clients and want them to come in and buy websites from us, we are also working away in the background trying to come up with ways that we can give our tools away so that other people can create websites (and more) using our tools.

As an example, our control panel, which we use for creating websites, is currently also shared out so a number of resellers throughout Ireland can create websites themselves using the same tools we use (if you’re interested in this, contact Kae).

The reason we are doing this, is that we love working on the automation side of the equation. We love letting our clients edit their own sites whenever they want, and we love giving our resellers the ability to create and sell websites even if they are not web designers themselves.

We even love the people that are not using our CMS engine.

For those of you out there that already have websites from other providers, we are writing a suite of tools to give you added abilities on those sites – for example, imagine adding a Facebook-like chat to your website by adding a single line of code to your website? Or maybe adding a subscription tool that lets you send SMS messages to your visitors? It’s all on the way…

success vs comfort

Over the last week, I’ve had the opportunity to work with some people who are building a large-scale application for a company. I can’t say much about it other than the company is a “house-hold name” online, and you will probably come across the work in your own online life soon.

The architecture (which I was hired to look at and suggest improvements for) is interesting, but what interested me more were the people driving the work.

It seems to me that the more people there are in a company, the more specialised the language gets when managers speak to each other. There was a lot of talk of “crud”, “sprints”, “stand-up meetings”, etc., each of which seemed a bit over-abstracted to me – it seemed to me that these things are for the benefit of the managers, and that the work itself could be done quicker without them.

I wasn’t very comfortable with it. I prefer to get on with the work. Spending an hour in the morning discussing what I’m going to do, and an hour at the end of the day discussing what I got done that day seems to me like I’ve just spent 25% of the working day talking when I could have been working.

I’m used to building up a plan which may be a week or two long, then getting it done. Having to stop every few hours to explain to people what is really a bit too low-level to be of interest to them is just not my idea of fun. It is my opinion that a manager needs to know when a project is ready to be demoed or published, but to invite a manager to dig into the engine and examine the nuts and bolts seems a bit mad.

Facebook do the exact opposite, and I admire them for it. They allow the programmers free reign to do their work whatever way they need to, and are only interested in results – not whether the method followed was Agile, Waterfall, Scrum, or some other cycle.

Companies that follow certain coding philosophies do it with a specific goal in mind – they want to be rich, and quickly.

I’m kind of the opposite to that, as can be seen by my prices. While it would be nice to be rich, I really do not want to go far out of my way to obtain that goal. I am more interested in paying the bills and not being stressed, than in some mad rat-race to get paid twice as much as I’m currently getting.

And so my own philosophy is to work away on things, incrementally improving my products, and to never take risks. If I do well, then I will eventually make a tidy sum. If I don’t, then I will at least be comfortable.

And that, for me, is what’s important. I prefer to go home at 5pm to a comfortable couch and a book, than to have a load of money in the bank and a stressful life.

first two lessons learned

In late October last year, I applied to join a “start your own business” course at the Monaghan County Enterprise Board. The course was unfortunately delayed until the beginning of February.

I wish it had started on time, though, as I’ve had to learn a few lessons the hard way. The most important two are that people generally pay late (it’s just a fact of life), and that cash is king. These lessons were part of week two’s course, where we got to analyse a test-case budget. I could see my own budget peering out at me through those numbers!

So, what do these lessons mean?

People generally pay late

When you write down in your budget that you will do the work one week and get paid the next, you are being very very optimistic. People will have all sorts of reasons that they cannot pay you on time. Their own bills need to be paid, they haven’t the money, or they will simply not pay because the job is not yet finished to their satisfaction (even if what’s missing is something you’re waiting on them to provide).

This happens with almost all customers where a service is what’s being paid for. When you buy a TV, you expect to have to pay up front before you’ve even turned it on, but if you’re buying a website, you expect to be able to pay when you are happy that the site is up to your expectations.

This is unfortunate for the poor web developer, who needs the money. Ah well, that’s one lesson learned!

How to avoid it? Difficult to say. Three suggestions spring to mind, and I am trying them out myself at the moment.

  • Offer a discount if the entire bill is paid within 30 days. This is whether the job is completed or not. And make sure you do complete the job! Your reputation is important.
  • Offer terms where the balance should be paid at regular intervals every few weeks. For example, after the deposit is paid (you are already asking for a deposit, right?), insist that the balance be paid in regular chunks every two weeks.
  • Ask the client to sign a contract where they agree to pay the entire balance within a set duration (30 days, say) of work starting. In return, you have as part of the contract that you will complete your part of the bargain within half that time.

I’d love to hear any comments on these?

Cash Is King

I’ve also been bitten by this one.

Let’s say you’re lucky enough to have 3 jobs running. They have each paid deposits. Your work is now completed, and you are owed a thousand or more.

Sounds nice, doesn’t it?

Well, let’s say a week rolls by. The deposits have all been spent on bills, and you are still waiting for the clients to pay. You have now got an empty bank account.

Then you need to pay yourself so you can go buy food. The clients still owe you a thousand, which means you have money on paper, but you don’t have any cash.

In this situation, you’re screwed. You need cash to survive. There is no point having huge contracts on if there is no regular income.

Again, a possible solution to this is to ask that your clients sign contracts meaning they must pay you either within a set amount of time, or in regular lump sums.

WebME plan for 2011

WebME is the CMS that KV Sites uses. It’s free to download and install, if you have a server of your own and the technical ability, but to get the best experience, it’s better to have someone host it and manage it for you (that’s us).

As WebME is the “core” application of KV Sites, it makes a lot of sense to always work on improving it.

Every now and then, I look at the “competition” out there, and see how they’re doing things – what ideas can I “steal”, or improve – what are they doing that I think is clumsy or could be easier for the user – what are they doing that’s awesome.

This morning, I looked at a number of different CMSes, including CMS Made Simple, Drupal, Joomla, Xoops, Plone, WordPress, ModX, Concrete5, Exponent, Website Baker, and Mambo.

I carefully read through their plugins/modules lists, and made up a spreadsheet of what I found.

After sorting and counting, I’ve found the following are the most popular (as in, most widely available) types of feature: news/rss, photo galleries, polls, calendars, guestbooks/comments, searches, online stores, and stats.

I’ve counted 40 different feature types in total, but those are the most popular.

Doing this has led me to the following plan for the next few months, to improve WebME:

  • I need to create more publically available themes.
  • More RSS integration in WebME. for news, forums, comments, etc.
  • Examine exactly what are the top CMSes doing with galleries, polls, etc, and make sure my own versions are at least as capable.
  • I need to publicise WebME more, so I get developers interested in writing plugins for it.
  • Ease of use is much more important than feature-richness. WordPress is a perfect example of ease of use, whereas Joomla and Drupal are very very frightening when first looked at!
  • More in-depth look at that spreadsheet. I want to drill down into what features of each plugin/module are popular and make sure I handle them all.

While I do have plugins for each of those popular features I mentioned, there are a number of features I found mentioned that WebME still doesn’t do. I’ll write a plugin for each of the following:

  • Social site integration – facebook, twitter, etc.
  • Chat. Users should be able to chat with each other, or to support staff.
  • Blogs are very popular at the moment, but WebME is not designed around that. A goal for this year is to move this blog to WebME, without losing any functionality at all.
  • Newsletters, event booking, content-scheduling, classified ads, private user messages, surveys, page ratings, FAQs, pagination for long articles, article previews, trashing instead of deleting, wiki – these are all features WebME does not yet do. I will tackle one per week.

All in all, 2011 is going to be a great year for WebME – by the end of the year, I will be able to demonstrate it as being a real contender in the CMS market. It is already very capable, but by the end of the year, I want people that I have never spoken to to really seriously consider WebME when they decide what CMS to use for their websites.