Skip to main content

Edge computing with Cloudflare Workers and CouchDB


The “Speed” problem

I’ve developed a Hong Kong based website (domestic parenting website) https://popa.help for my client in 2019. The site loads within 5 secs in Hong Kong. However, after I moved to Canada, the site loads more than 12 secs. It’s slower more than 100%. I suddenly realize that network latency is important for international users.

To solve this obstacle, I proposed Edge Computing to the client. And I’ve developed several POCs (proof-of-concept). Finally, I chose Cloudflare Workers and CouchDB. The result of the new site is 444% faster. The new implementation is mentioned in the below sections.

Introducing Cloudflare Workers

The existing site is running on AWS Lambda. Obviously, it is not an edge computing platform. Though it has Lambda@Edge, Cloudflare Workers are still easy to develop, running faster and lower cost than others. As mentioned in Cloudflare Wiki, Cloudflare is a web-known CDN provider that had installed over 200 data centers around the world. In Mar 2018, Cloudflare announced Workers (Cloudflare Workers). Cloudflare Workers is a serverless platform that runs Javascript Service Workers at the Edge data centers. The Cloudflare Workers webapp can respond to the user where it is the closest data center. This solves the network latency when request our current domestic webapp (based in HK) from anywhere in the world.

Unlike AWS Lambda, Cloudflare Workers runs Javascript using Google V8 (Javascript engine in Google Chrome) rather than NodeJS. V8 executes Javascript inside sandbox instead NodeJS executes inside a VM or container. In shorts, Cloudflare Workers performance is higher but I need to write more codes to fill the lack of NodeJS/ExpressJS.

Another killer feature of Cloudflare Workers is the deployment speed. When deploying the codes which can be less than 30 sec to globally 200+ cities.

Why CouchDB?

CouchDB is one of NoSQL databases like MongoDB. CouchDB’s distinguishing features is its couch replication protocol. It makes CouchDB clustering can perform multi-master replication (MongoDB can only perform master-slave cluster). Multi-master clustering is a perfect fit for Edge Computing. When a record is updated in any server, CouchDB auto-sync to others less than 1 sec. So, all users are seeing the same data no matter anywhere in the world.

You may wonder why not use Cloudflare Workers KV? After I fully tested, the KV has a limitation. The most serious obstacle is KV handling continuously reads/writes very slow. Much slower than CouchDB.

What’s the benefit to mix Cloudflare Workers with CouchDB?

As mentioned above, Cloudflare Workers is an Edge Computing serverless platform. It does a very useful feature: return the “ip-to-country code” in the request header ‘cf’ Object. That’s the country code of the user locating. I developed a simple mapping function to calculate the closest database server. The flow of the function as below:

I proposed the client to setup a CouchDB cluster. The 3 CouchDB servers were in Hong Kong, U.S. and U.K. The geographic locations of the CouchDB servers:

Below is the example cities map to CouchDB server results:
Example citiesContinentDestination server
Hong KongAsiaH.K. CouchDB server 
VancouverNorth AmericaU.S. CouchDB server
NetherlandEuropeU.K. CouchDB server
TokyoAsiaH.K. CouchDB server
SydneyOceaniaH.K. CouchDB server
Sao PauloSouth AmericaU.S. CouchDB server
South AfricaAfricaU.K. CouchDB server
TaiwanAsiaH.K. CouchDB server


After the infrastructure is setup, it’s time to develop Cloudflare Workers webapp.

Develop the webapp step by step

Prerequisite

  1. You’ll need a Cloudflare account
  2. Access to the API keys for that account
  3. Install the Wrangler command line interface

Coding

1. Create the project (naming edge-cloudflareworkers-couchdb):
$ wrangler generate edge-cloudflareworkers-couchdb https://github.com/cloudflare/worker-template-router
$ cd edge-cloudflareworkers-couchdb

2. Edit the wrangler.toml
name = "edge-cloudflareworkers-couchdb"
type = "webpack"
account_id = "[your cloudflare workers account id]"
workers_dev = true
route = ""
zone_id = "[your cloudflare workers zone id]"

Please refer to the official full instructions of Cloudflare Workers

3. Add the country code detection function

Open the source code index.js and modify function handleRequest() and add findNearestCdb(). The result is stored in this file.

4. Start the preview server
$ wrangler preview --watch

The program outputs

Screen 1: Get my current country code:

Screen 2: Closest server found:

Conclusion

The goal is SPEED. I am helping my client to implement edge computing. The site loading speed is reduced from 11.11s to 2.5s (444% faster):

The implementation is still work in progress. It is not only improving the backend but also the frontend. GraphQL and lit-element (web components) are implementing on the frontend to reduce network traffic and lightning-fast templating. Please leave a comment when you want me to show that 😉


Site speeds comparison (you’ll see the difference especially when you’re out of Hong Kong):

Labels:
Location: Vancouver, BC, Canada

Comments

Post a Comment

Popular posts from this blog

Create An Online Store Theme Used By MyCMS

MyCMS is an open-source Content Management System to generate static online shop website. You can use my hosting to input your products, or you can download the source codes and host it in your own server (running NodeJS). Please refer to my Github repo for the detailed installation instructions. This blog is a step-by-step tutorial that shows you how to create an online-shop theme. In this tutorial, it’s using my hosting to input the shop details and products. If you’re hosting the MyCMS by yourself, just change the domain name to yours will do. Introducing MyCMS Before making the theme, you’ll need to use MyCMS to configure the demo shop and input two demo products. MyCMS generates a static website via a theme. The generated static website is NO server program required. You can put the website files (HTML/CSS/JS) to any CDN, hosting. Shop Configuration You must prepare below settings Before using MyCMS: Setting Description Example Store name Your store name will be displayed in t...
1 comment
Read more

How I make a web-components based dynamic Javascript page at the top of Google Search

Introduction Everybody wants their website shown at the first position of Google search. SEO (Search Engine Optimization) is a big topic. I just helped my client's website shows the database records at the top search rankings (at least several Chinese generic keywords). See the three example questions all are listed at top ranking: Website background: My client's website  popa.qa  is a traditional Chinese Q&A site that lets members ask and answer questions. All answers and questions are storing in the database server. Step 1: Create The Project This blog illustrates the problems and the steps to fix that with project source codes. Below is the description of the basic project: NodeJS backend (server.js) Develop an API ( /get-database-records ) to simulate getting database records Web-components frontend (index.html) An example component IndexPage  make use of  LitElement to render database records To start the server type: npm start Then I check the webpag...
1 comment
Read more

Build A Simple Sales System using Google Drive

Google Drive is a free-to-use cloud service with a 15 GB limit storage. 15 GB is large enough to store documents, and "program" too. Yes, it's correct. We can program Google Drive. This blog is a step-by-step tutorial to show you how to build a simple P.O.S. system on Google Drive. Create a Google Sheets to define the products 1. Login to Google Drive using your Gmail account and create a Google Sheets: 2. Input the products in the Google Sheets Create a Google Forms for sale input 1. Create a new Google Form 2. Edit the form and the first field Note that leave the option field unchanged. We'll assign it programmatically in below 3. Add a quantity field 4.  Add a last field for the price input How to copy data from Google Sheets to Google Forms? We need to add a script to do that. Google developed Google App Script (GAS) for us to achieve that. Now go back to the Google Sheet we just created above. Then enter to the script editor You'll see the below screen. Then ...
Post a Comment
Read more